-- Logs begin at Thu 2022-09-08 12:46:52 UTC, end at Thu 2022-09-08 14:46:26 UTC. -- Sep 08 13:44:32.402852 dv-cinder-856460-1-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 08 13:44:32.412355 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 08 13:44:32.415920 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 8 13:44:32 2022] *** Sep 08 13:44:32.415920 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 08 13:44:32.415920 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 08 13:44:32.415920 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: nodename: dv-cinder-856460-1-iscsi Sep 08 13:44:32.415920 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: machine: x86_64 Sep 08 13:44:32.415920 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: clock source: unix Sep 08 13:44:32.415920 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: pcre jit disabled Sep 08 13:44:32.415920 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: detected number of CPU cores: 4 Sep 08 13:44:32.415920 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: current working directory: / Sep 08 13:44:32.415920 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: detected binary path: /usr/bin/uwsgi-core Sep 08 13:44:32.415920 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: your processes number limit is 47680 Sep 08 13:44:32.415920 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: your memory page size is 4096 bytes Sep 08 13:44:32.415920 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: detected max file descriptor number: 2048 Sep 08 13:44:32.415920 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: lock engine: pthread robust mutexes Sep 08 13:44:32.415920 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: thunder lock: enabled Sep 08 13:44:32.415920 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 08 13:44:32.415920 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 08 13:44:32.453893 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: Python main interpreter initialized at 0x564175c719c0 Sep 08 13:44:32.453893 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: python threads support enabled Sep 08 13:44:32.453893 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: your server socket listen backlog is limited to 100 connections Sep 08 13:44:32.453893 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: your mercy for graceful operations on workers is 90 seconds Sep 08 13:44:32.454303 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: mapped 403077 bytes (393 KB) for 2 cores Sep 08 13:44:32.454389 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: *** Operational MODE: preforking *** Sep 08 13:44:32.454590 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: *** uWSGI is running in multiple interpreter mode *** Sep 08 13:44:32.454590 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: spawned uWSGI master process (pid: 112546) Sep 08 13:44:32.455332 dv-cinder-856460-1-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 08 13:44:32.455940 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: spawned uWSGI worker 1 (pid: 112547, cores: 1) Sep 08 13:44:32.456225 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: spawned uWSGI worker 2 (pid: 112548, cores: 1) Sep 08 13:44:32.456335 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 08 13:44:33.120373 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 08 13:44:33.122550 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 08 13:44:34.213104 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:44:34.213104 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: from cryptography.utils import int_from_bytes Sep 08 13:44:34.213819 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:44:34.213819 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: from cryptography.utils import int_from_bytes Sep 08 13:44:34.213969 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:44:34.213969 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: from cryptography.utils import int_from_bytes Sep 08 13:44:34.214382 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:44:34.214382 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: from cryptography.utils import int_from_bytes Sep 08 13:44:36.770528 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 08 13:44:36.771111 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 08 13:44:36.771301 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 08 13:44:36.771659 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 08 13:44:36.771756 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 08 13:44:36.772107 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 08 13:44:36.772197 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 08 13:44:36.772315 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.772463 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 08 13:44:36.772594 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.772826 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 08 13:44:36.772919 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.773083 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 08 13:44:36.773218 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.773469 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.773562 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.773794 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.773887 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.774117 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.774250 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.774459 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.774563 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.774788 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.774887 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.775056 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.775211 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.775367 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.775474 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.775691 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-856460-1-iscsi {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.775797 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.776009 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.776104 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.776267 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.776362 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.776579 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.776675 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.776903 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-856460-1-iscsi {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.776998 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.777203 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.777303 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.777517 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.777611 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.777816 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.777910 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.778131 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.778384 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.778501 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.778758 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.778849 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.779059 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.779136 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.779371 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.779557 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.779703 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.779865 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.780014 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.780169 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.780319 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.780476 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.780633 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.780808 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.780937 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.781120 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.781257 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.781428 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.781567 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.781748 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.781946 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-856460-1-iscsi {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.782044 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.782286 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.782396 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.782596 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.782704 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.782940 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.783115 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-856460-1-iscsi {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.783351 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.783454 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.783661 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.783756 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.783969 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.784067 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.784286 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.784374 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.784616 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.784710 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.784921 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.785016 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.785243 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.785337 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.785559 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.785666 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.785867 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.785961 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.786206 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.786313 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.786509 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.786607 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.786813 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.786909 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.787118 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.787257 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.787415 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.787516 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.787722 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.787816 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.788028 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.788123 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.788277 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.788375 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.788581 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.788675 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.788899 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.788992 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.789149 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.789325 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.789476 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.789568 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.789753 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.789848 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.790002 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.790092 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.790352 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.790442 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.790632 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.790726 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.790882 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.790973 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.791198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.791293 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.791516 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.791627 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.791825 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.791923 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.792127 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.792220 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.792429 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.792525 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.792730 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.792825 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.793031 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.793118 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.793333 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.793438 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.793636 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.793731 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.793942 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.794032 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.794269 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.794362 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.794587 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.794732 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.794974 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.795062 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.25 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.795219 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.25 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.795350 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.795523 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.795620 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.795856 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.795952 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.25 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.796169 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.796264 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.25 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.796472 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.796558 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.796785 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.796884 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.797100 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.797190 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.797400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.797486 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.797699 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.797793 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.798006 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.798096 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.798863 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.798956 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.799174 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.799283 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.799481 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.799613 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.799709 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.799801 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.800012 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.800113 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.800313 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.800402 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.800628 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.800723 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.800929 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.801027 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.801232 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.801320 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.801531 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.801625 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.801826 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.801968 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.802128 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.802318 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.802453 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.802540 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.802754 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.802845 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.803054 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.803141 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.803355 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.803443 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.803656 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.803745 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.803952 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.804051 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.804254 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.804340 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.804552 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.804647 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.804849 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.804935 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.805154 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.805248 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.805458 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.805547 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.805757 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.805844 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.806057 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.806181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.806387 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.806529 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.806687 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.806786 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.806988 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.807092 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.807285 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.807373 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.807583 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.807676 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.807896 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.807983 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.808195 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.808285 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.808499 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.808585 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.808808 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.808894 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.809107 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.809194 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.809406 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.809624 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.809753 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.809854 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.810000 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.810087 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.810325 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.810417 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.810635 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.810745 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.810931 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.811023 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.811226 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.811324 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.811522 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.811619 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.811820 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.811909 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.812117 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.812207 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.812412 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.812508 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.812705 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.812792 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.813002 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.813090 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.813293 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.813390 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.813588 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.813675 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.813887 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.814031 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.814950 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.815094 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.815204 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.815309 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.815440 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.815578 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.815744 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.815845 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.816048 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.816138 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.816351 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.816446 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.816667 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:44:36.816766 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.817000 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.817088 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.817333 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.817433 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.817660 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.817756 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.817971 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.818058 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.818301 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.818407 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.818619 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.818708 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.818942 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.819038 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.819249 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.819348 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.819555 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.819648 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.819864 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.819961 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.820170 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.820264 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.820478 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.820573 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.820789 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.820889 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.821104 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.821203 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.821415 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.821510 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.821731 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.821819 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.822043 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.822246 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.822380 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.822481 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.822698 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.822831 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.823010 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.823100 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.823315 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.823402 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.823638 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.823726 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.823956 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.824058 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.824264 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.824352 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.824582 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.824678 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.824899 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.824988 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.825210 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.825313 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.825518 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.825648 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.825825 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.825922 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.826158 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.826249 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.826473 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.826577 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.826782 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.826878 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.827099 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.827187 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.827410 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.827511 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.827723 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.827811 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.828028 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.828114 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.828338 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.828435 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.828653 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.828741 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.828963 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.829058 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.829269 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.829369 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.829617 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.829713 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.829916 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.830012 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.830257 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.830361 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.830573 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.830668 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.830883 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.830978 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.831196 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.831286 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.831506 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.831594 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.831815 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.831903 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.832123 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.832259 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.832424 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.832520 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.832726 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.832814 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.833027 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.833116 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.833328 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.833415 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.833626 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.833724 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.833927 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.834015 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.834276 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.834377 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.834582 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.834684 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.834911 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.835000 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.835214 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.835302 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.835522 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.835622 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.835830 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.835927 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.836138 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.836233 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.836441 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.836547 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.836751 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.836845 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.837057 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.837144 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.837359 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.837456 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.837672 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.837760 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.837979 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.838073 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.838311 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.838413 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.838633 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.838720 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.838936 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.839023 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.839240 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.839338 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.839553 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.839639 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.839855 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.839949 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.840162 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.840263 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.840474 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.840561 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.840790 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.840889 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.841096 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.841194 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.841410 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.841499 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.841719 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.841816 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.842040 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.842129 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.842376 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.842476 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.842692 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.842805 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.843021 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.843110 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.843350 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.843445 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.843653 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.843754 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.843955 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.844051 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.844266 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.844355 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.844574 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.844670 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.844875 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.844962 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.845175 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.845261 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.845503 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.845631 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.845808 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.845896 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.846112 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.846266 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.846440 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.846543 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.846740 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.846828 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.847041 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.847136 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.847343 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.847433 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.847642 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.847730 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.847941 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.848035 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.848242 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.848332 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.848548 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.848635 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.848846 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.848950 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.849146 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.849235 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.849446 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.849542 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.849786 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.849883 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.850089 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.850293 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.850452 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.850550 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.850762 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.850862 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.851078 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.851166 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.851389 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.851476 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.851700 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.851801 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.852010 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.852098 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.852317 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.852441 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.852629 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.852721 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.852935 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.853022 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.853243 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.853329 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.853550 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.853638 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.853858 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.853945 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.854184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.854286 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.854517 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.854625 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.854806 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.854894 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.855118 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.855206 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.855439 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.855566 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.855744 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.855840 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.856060 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.856152 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.856414 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.856522 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.856688 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.856785 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.857003 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.857100 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.857311 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.857405 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.857620 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.857708 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.857930 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.858026 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.858262 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.858352 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.858572 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.858673 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.858881 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.858979 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.859190 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.859288 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.859495 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.859583 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.859803 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.859904 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.860110 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.860205 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.860415 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.860502 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.860715 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.861557 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.861799 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.861887 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.862114 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.862260 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.862482 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.862621 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.862752 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.862841 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.863061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.863157 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.863382 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.863478 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.863692 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.863816 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.863999 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.864096 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.864308 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.864404 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.864613 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.864702 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.864919 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.865016 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.865227 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.865314 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.865532 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.865627 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.865836 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.865933 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.866183 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.866276 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.866467 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.866555 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.866779 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.866880 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.867085 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.867174 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.867387 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.867483 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.867686 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.867782 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.867993 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.868089 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.868310 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.868396 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.868612 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.868702 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.868922 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.869010 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.869244 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.869341 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.869544 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.869632 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.869845 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.869934 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.870185 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.870317 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.870491 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.870583 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.870788 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.870875 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.871090 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.871185 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.871391 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.871483 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.871689 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.871778 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.872028 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.872124 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.872363 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.872465 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.872707 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.872796 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.873019 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.873116 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.873325 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.873415 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.873641 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.873729 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.873961 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.874057 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.874295 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.874387 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.874609 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.874698 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.874919 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.875021 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.875226 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.875314 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.875538 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.875635 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.875850 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.875991 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.876157 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.876252 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.876511 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.877218 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.877326 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.877419 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.877636 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.877725 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.877946 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.878042 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.878283 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.878375 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.878587 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.878676 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.878897 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.878984 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.879220 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.879310 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.879553 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.879791 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.879881 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.880126 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.880226 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.880444 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.880714 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.880803 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.881049 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.881147 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.881366 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.881468 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.881684 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.881775 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.882024 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.882128 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.882365 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.882455 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.882672 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.882814 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.882984 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.883072 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.883289 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.883376 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.883594 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.883692 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.883902 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.883990 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.884202 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.884302 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.884505 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.884594 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.884816 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.884905 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.885119 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.885216 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.885420 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.885506 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.885725 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.885859 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.886027 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.886198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.886353 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.886442 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.886662 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.886759 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.886964 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.887061 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.887266 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.887354 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.887566 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.887699 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.887869 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.887959 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.888179 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.888266 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.888482 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.888571 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.888784 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.888882 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.889093 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.889182 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.889392 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.889486 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.889695 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.889784 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.889999 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.890095 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.890366 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.890458 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.890671 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.890763 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.890998 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.891095 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.891302 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.891400 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.891624 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.891714 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.891926 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.892015 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.892256 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.892356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.892560 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.892650 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.892866 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.892962 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.893171 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.893343 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.893471 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.893559 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.893779 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.893867 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.894085 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.894192 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.894412 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.894521 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.894738 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.894850 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.895038 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.895128 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.895345 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.895443 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.895646 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.895735 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.895959 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.896056 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.896266 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.896406 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.896567 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.896656 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.896868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.896958 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.897173 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.897275 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.897477 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.897574 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.897776 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.897870 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.898082 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.898252 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.898407 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.898513 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.898714 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.898801 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.899018 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.899105 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.899319 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.899423 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.899620 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.899709 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.899925 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.900013 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.900229 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.900327 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.900531 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.900620 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.900841 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.900931 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.901150 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.901256 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.901451 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.901538 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.901753 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.901849 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.902063 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.902171 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.902389 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.902478 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.902692 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.902779 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.902998 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.903087 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.903301 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.903390 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.903622 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.903729 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.903920 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.904010 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.904229 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.904324 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.904531 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.904629 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.904833 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.904922 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.905134 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.905236 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.905440 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.905538 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.905742 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.905833 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.906043 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.906187 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.906374 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.906642 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.906730 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.906813 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.906961 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.907126 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.907281 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.907653 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.908100 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.908197 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.908289 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.908372 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.908453 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.908543 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.908682 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.908989 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.909443 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.909589 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.909687 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.909772 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.909860 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.909944 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.910025 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.910115 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.910320 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.910413 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.910628 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.910725 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.910990 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.911078 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.911299 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.911388 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.911601 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.911689 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.911915 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.912002 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.912226 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.912315 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.912533 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.912628 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.912835 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.912932 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.913135 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.913224 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.913450 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.913587 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.913760 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.913861 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.914066 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.914173 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.914392 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.914492 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.914696 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.914798 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.915016 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.915118 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.915316 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.915404 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.915622 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.915720 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.915932 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.916028 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.916239 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.916335 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.916545 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.916637 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.916845 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.916941 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.917152 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.917238 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.917462 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.917560 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.917768 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.917862 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.918073 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.918200 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.918408 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.918505 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.918714 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.918841 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.919024 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.919111 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.919330 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.919419 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.919631 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.919719 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.919953 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.920041 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.920260 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.920348 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.920562 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.920650 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.920869 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.920974 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.921171 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.921260 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.921481 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.921568 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.921784 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.921947 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.922103 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.922225 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.922436 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.922532 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.922741 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.922840 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.923053 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.923181 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.923888 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.923994 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.924266 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.924355 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.924583 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.924671 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.924899 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.924986 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.925203 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.925292 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.925511 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.925620 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.925824 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.925933 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.926196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.926291 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.926478 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.926566 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.926784 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.926875 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.927142 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.927239 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.927453 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.927541 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.927775 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.927871 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.928079 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.928168 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.928382 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.928478 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.928684 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.928779 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.928995 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.929085 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.929294 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.929381 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.929608 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.929705 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.929912 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.930001 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.930307 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.930398 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.930632 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.930730 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.930935 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.931070 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.931257 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.931345 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.931643 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.931731 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.931966 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.932054 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.932267 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.932354 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.932600 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.932697 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.932906 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.933005 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.933217 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.933314 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.933526 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.933615 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.933849 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.933950 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.934189 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.934282 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.934525 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.934623 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.934819 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.934918 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.935124 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.935212 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.935444 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.935532 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.935761 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.935858 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.936069 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.936157 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.936383 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.936480 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.936758 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.936849 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.937087 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.937186 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.937397 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.937493 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.937711 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.937799 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.938032 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.938120 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.938362 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.938466 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.938667 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.938786 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.939568 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.939667 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.939890 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.939988 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.940200 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.940291 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.940523 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.940612 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.940827 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.940918 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.941137 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.941233 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.941440 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.941529 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.941747 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.941836 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.942051 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.942194 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.942380 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.942479 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.942690 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.942780 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.942991 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.943078 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.943333 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.943428 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.943605 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.943697 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.943910 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.944007 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.944231 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.944334 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.944533 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.944629 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.944854 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.944950 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.945164 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.945266 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.945469 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.945561 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.945773 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.945870 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.946076 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.946200 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.946425 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.946514 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.946737 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.946833 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.947043 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.947142 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.947347 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.947443 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.947651 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.947741 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.947974 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.948064 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.948290 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.948385 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.948614 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.948703 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.948929 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.949019 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.949236 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.949325 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.949551 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.949640 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.949876 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.949966 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.950210 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.950302 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.950491 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.950580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.950798 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.950890 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.951100 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.951229 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.951401 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.951501 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.951703 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.951836 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.952010 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.952099 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.952311 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.952409 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.952613 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.952709 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.952913 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.953001 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.953212 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.953309 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.953520 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.953611 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.953822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.953911 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.954124 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.954245 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.954449 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.954536 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.954777 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.954867 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.955080 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.955172 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.955403 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.955534 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.955710 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.955799 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.956012 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.956101 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.956315 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.956406 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.956616 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.956706 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.956934 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.957024 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.957247 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.957336 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.957560 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.957649 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.957867 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.957956 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.958196 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.958287 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.958511 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.958612 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.958825 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.958923 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.959133 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.959222 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.959443 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.959541 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.959759 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.959847 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.960070 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.960158 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.960385 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.960590 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.960681 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.960916 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.961005 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.961227 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.961372 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.961538 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.961628 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.962109 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.962263 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.962373 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.962577 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.962667 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.962793 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.962930 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.963088 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.963178 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.963398 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.963488 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.963706 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.963802 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.964029 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.964117 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.964351 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.964468 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.964670 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.964759 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.965000 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.965099 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.965314 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.965405 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.965628 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.965900 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.965979 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.966073 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.966353 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.966447 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.966655 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.966746 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.966975 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.967076 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.967294 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.967384 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.967610 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.967699 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.967926 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.968015 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.968241 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.968332 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.968553 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.968656 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.968864 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.968954 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.969175 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.969265 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.969488 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.969588 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.969801 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.969890 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.970110 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.970279 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.970472 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.970571 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.970785 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.970872 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.971097 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.971185 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.971412 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.971510 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.971724 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.971814 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.972035 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.972123 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.972348 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.972437 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.972660 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.972748 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.972993 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.973207 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.973310 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.973523 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.973612 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.973855 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.973943 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.974202 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.974404 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.974568 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.974857 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.974947 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.975183 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.975293 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.975506 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.975594 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.975811 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.975900 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.976116 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.976214 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.976427 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.976524 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.976737 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.976834 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.977040 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.977139 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.977361 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.977464 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.977669 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.977757 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.977970 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.978066 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.978316 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.978466 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.978613 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.978753 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.978959 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.979148 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.979305 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.979500 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.979649 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.979822 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.979984 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.980129 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.980287 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.980455 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.980700 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.980865 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.981047 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.981208 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.981350 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.981658 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.981658 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.981897 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.981992 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.982073 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.982274 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.982379 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.982587 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.982687 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.982889 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.982978 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.983197 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.983361 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.983510 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.983676 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.983814 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.983904 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.984081 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.984171 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.984383 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.984485 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.984682 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.984770 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.984990 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.985086 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.985297 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.985394 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.985598 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.985689 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.986705 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.986819 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.987019 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.987110 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.987325 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.987413 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.987627 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.987716 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.987930 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.988019 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.988229 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.988326 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.988538 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.988634 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.988841 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.988939 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.989142 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.989230 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.989441 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.989529 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.989741 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.989839 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.990051 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.990189 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.990380 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.990483 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.990690 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.990779 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.990994 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.991084 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.991302 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.991398 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.991606 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.991697 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.991923 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.992012 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.992232 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.992400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.992551 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.992639 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.992855 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.992945 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.993161 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.993263 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.993479 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.993569 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.993799 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.993929 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.994123 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.994363 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.994499 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.994704 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.994794 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.995029 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.995137 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.995366 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.995457 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.995699 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.995912 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.996014 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.996248 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.996340 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.996570 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.996666 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.996902 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.996990 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.997215 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.997304 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.997528 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.997607 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.997837 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.997924 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.998176 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.998284 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.998484 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.998604 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.998794 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.998923 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.999103 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.999239 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.999412 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.999551 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.999723 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:36.999862 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.000031 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.000171 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.000345 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.000482 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.000653 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.000793 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.000961 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.001182 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.001271 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.001557 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.002204 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.002407 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.002498 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.002720 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.002809 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.003034 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.003133 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.003346 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.003443 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.003655 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.003745 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.003974 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.004113 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.004268 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.004416 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.004563 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.004711 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.004851 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.004983 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.005184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.005344 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.005488 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.005635 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.005789 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.005934 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.006092 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.006347 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.006493 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.006758 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.006914 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.007068 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.007159 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.007397 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.007488 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.007710 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.007810 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.008015 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.008104 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.008339 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.008665 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.008764 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.008979 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.009080 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.009301 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.009391 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.009615 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.009706 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.009942 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.010031 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.010301 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.010393 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.010630 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.010722 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.010946 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.011037 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.011250 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.011338 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.011553 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.011652 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.011857 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.011955 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.012168 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.012256 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.012486 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.012708 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.012804 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.012874 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.013086 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.013187 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.013388 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.013476 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.013696 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.013784 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.014000 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.014097 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.014330 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.014438 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.014635 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.014724 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.014938 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.015036 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.015243 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.015332 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.015542 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.015630 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.015843 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.015933 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.016142 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.016229 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.016441 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.016528 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.016741 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.016837 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.017042 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.017150 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.017380 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.017469 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.017684 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.017775 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.017988 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.018077 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.018314 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.018402 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.018615 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.018705 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.018916 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.019012 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.019224 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.019320 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.019526 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.019616 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.019830 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.019926 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.020131 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.020234 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.020434 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.020523 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.020737 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.020835 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.021068 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.021165 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.021402 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.021499 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.021718 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.021807 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.022033 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.022285 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.022375 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.022467 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.022689 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.022778 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.023000 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.023090 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.023314 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.023403 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.023627 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.023715 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.023940 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.024039 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.024251 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.024341 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.024560 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.024648 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.024890 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.024984 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.025199 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.025294 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.025509 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.025596 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.025821 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.025910 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.026158 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.026251 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.026468 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.026576 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.026787 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.026883 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.027100 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.027197 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.027404 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.027501 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.027757 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.027850 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.028037 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.028125 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.028496 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.030484 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.030573 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.030656 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.030747 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.030838 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.030930 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.031021 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.031105 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.031186 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.031267 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.031356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.031451 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.031541 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.031630 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.031720 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.031809 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.031891 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.032028 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.032116 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.032198 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.032279 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.032360 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.032449 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.032532 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.032613 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.032713 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.032787 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.032856 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.032945 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.033072 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.033161 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.033370 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.033498 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.033688 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.033792 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.034003 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.034111 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.034329 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.034441 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.034566 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=112548) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 08 13:44:37.034759 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.035105 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.035435 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.035750 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.036063 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.036368 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:44:37.036610 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=112547) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 08 13:44:37.188752 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO dbcounter [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] Registered counter for database nova_api Sep 08 13:44:37.191493 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO dbcounter [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] Registered counter for database nova_api Sep 08 13:44:37.208642 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_db.sqlalchemy.engines [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112548) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:44:37.210640 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG oslo_db.sqlalchemy.engines [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112547) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:44:37.215311 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG dbcounter [-] [112548] Writer thread running {{(pid=112548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:44:37.217069 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG dbcounter [-] [112547] Writer thread running {{(pid=112547) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:44:37.274832 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.context [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf(cell1) {{(pid=112548) load_cells /opt/stack/nova/nova/context.py:464}} Sep 08 13:44:37.275606 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.context [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf(cell1) {{(pid=112547) load_cells /opt/stack/nova/nova/context.py:464}} Sep 08 13:44:37.278884 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:44:37.279345 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:44:37.279647 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG oslo_concurrency.lockutils [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:44:37.280118 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG oslo_concurrency.lockutils [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=112547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:44:37.280902 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:44:37.281638 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:44:37.281757 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG oslo_concurrency.lockutils [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=112547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:44:37.282077 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:44:37.282408 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG oslo_concurrency.lockutils [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:44:37.283449 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG oslo_concurrency.lockutils [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:44:37.284854 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:44:37.285442 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG oslo_concurrency.lockutils [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=112547) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:44:37.286862 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO dbcounter [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] Registered counter for database nova_cell0 Sep 08 13:44:37.287463 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO dbcounter [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] Registered counter for database nova_cell0 Sep 08 13:44:37.290953 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO dbcounter [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] Registered counter for database nova_cell1 Sep 08 13:44:37.291415 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO dbcounter [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] Registered counter for database nova_cell1 Sep 08 13:44:37.298205 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_db.sqlalchemy.engines [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112548) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:44:37.300030 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_db.sqlalchemy.engines [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112548) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:44:37.302775 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG oslo_db.sqlalchemy.engines [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112547) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:44:37.305131 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG oslo_db.sqlalchemy.engines [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112547) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:44:37.305970 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG dbcounter [-] [112548] Writer thread running {{(pid=112548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:44:37.310157 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG dbcounter [-] [112547] Writer thread running {{(pid=112547) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:44:37.311391 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG dbcounter [-] [112548] Writer thread running {{(pid=112548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:44:37.312886 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG dbcounter [-] [112547] Writer thread running {{(pid=112547) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:44:37.458247 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO dbcounter [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] Registered counter for database nova_cell0 Sep 08 13:44:37.460249 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO dbcounter [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] Registered counter for database nova_cell0 Sep 08 13:44:37.467381 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_db.sqlalchemy.engines [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112548) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:44:37.471494 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG oslo_db.sqlalchemy.engines [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=112547) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:44:37.473794 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG dbcounter [-] [112548] Writer thread running {{(pid=112548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:44:37.476864 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG dbcounter [-] [112547] Writer thread running {{(pid=112547) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:44:37.910711 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: WARNING keystonemiddleware.auth_token [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 08 13:44:37.917959 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: WARNING keystonemiddleware.auth_token [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 08 13:44:37.954365 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: WARNING keystonemiddleware.auth_token [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 08 13:44:37.963937 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: WARNING keystonemiddleware.auth_token [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 08 13:44:38.275968 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: WARNING keystonemiddleware.auth_token [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 08 13:44:38.281987 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: WARNING keystonemiddleware.auth_token [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 08 13:44:38.287914 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x564175c719c0 pid: 112548 (default app) Sep 08 13:44:38.289990 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:44:38.291214 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] 10.222.0.25 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001780 Sep 08 13:44:38.291612 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 788 bytes} [Thu Sep 8 13:44:38 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:44:38.338015 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: WARNING keystonemiddleware.auth_token [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 08 13:44:38.344000 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: WARNING keystonemiddleware.auth_token [None req-eac8ebd5-e559-445b-a1e3-c0fab3450c49 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 08 13:44:38.349306 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x564175c719c0 pid: 112547 (default app) Sep 08 13:46:06.193226 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:46:06.195013 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002314 Sep 08 13:46:06.195635 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 2/2] 10.222.0.25 () {54 vars in 931 bytes} [Thu Sep 8 13:46:06 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:46:06.213107 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG dbcounter [-] [112547] Writing DB stats nova_api:SELECT=3 {{(pid=112547) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:06.213884 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG dbcounter [-] [112547] Writing DB stats nova_cell1:SELECT=3 {{(pid=112547) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:06.214899 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG dbcounter [-] [112547] Writing DB stats nova_cell0:SELECT=3 {{(pid=112547) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:06.215724 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG dbcounter [-] [112547] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=112547) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:07.154624 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-1181de51-bd15-4a33-b20f-f366cc7f1892 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:46:07.189129 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-1181de51-bd15-4a33-b20f-f366cc7f1892 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 0.986151 Sep 08 13:46:07.189874 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 1/3] 10.222.0.25 () {60 vars in 1292 bytes} [Thu Sep 8 13:46:06 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 989 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:46:09.939629 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:46:09.940616 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-9a7d0f07-3c16-4a7f-b6a4-f36c6280e743 None None] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001395 Sep 08 13:46:09.941007 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 3/4] 10.222.0.25 () {54 vars in 931 bytes} [Thu Sep 8 13:46:09 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:46:10.445300 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-46042a7e-ab93-442a-94bb-574d007f87c4 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 256, "disk": 1, "swap": 0, "os-flavor-access:is_public": true, "vcpus": 1, "name": "cirros256", "id": "c1"}} {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:10.570764 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-46042a7e-ab93-442a-94bb-574d007f87c4 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.622215 Sep 08 13:46:10.570764 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 2/5] 10.222.0.25 () {64 vars in 1283 bytes} [Thu Sep 8 13:46:09 2022] POST /compute/v2.1/flavors => generated 407 bytes in 624 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:46:10.585133 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG dbcounter [-] [112548] Writing DB stats nova_api:SELECT=3 {{(pid=112548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:10.585932 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG dbcounter [-] [112548] Writing DB stats nova_cell0:SELECT=3 {{(pid=112548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:10.587864 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG dbcounter [-] [112548] Writing DB stats nova_cell1:SELECT=3 {{(pid=112548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:10.588592 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG dbcounter [-] [112548] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=112548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:10.799849 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-72f71fb0-21b9-45dd-b2fb-d18351dc5600 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:10.930357 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-72f71fb0-21b9-45dd-b2fb-d18351dc5600 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.350279 Sep 08 13:46:10.931173 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 4/6] 10.222.0.25 () {64 vars in 1336 bytes} [Thu Sep 8 13:46:10 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:46:13.779040 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-46042a7e-ab93-442a-94bb-574d007f87c4 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:46:13.780066 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-46042a7e-ab93-442a-94bb-574d007f87c4 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001511 Sep 08 13:46:13.780466 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 3/7] 10.222.0.25 () {54 vars in 931 bytes} [Thu Sep 8 13:46:13 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:46:14.549424 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-70be2068-7a9a-4ac8-89ae-3062f068faba admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "disk": 5, "ram": 512, "vcpus": 1, "id": "d1", "name": "ds512M", "rxtx_factor": 1.0, "swap": 0, "os-flavor-access:is_public": true}} {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:14.677081 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-70be2068-7a9a-4ac8-89ae-3062f068faba admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.889984 Sep 08 13:46:14.677642 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 5/8] 10.222.0.25 () {64 vars in 1283 bytes} [Thu Sep 8 13:46:13 2022] POST /compute/v2.1/flavors => generated 404 bytes in 891 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:46:14.690198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-310ea2a6-fc8a-4e39-ada3-3fca0ad3e9f8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:14.790073 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-310ea2a6-fc8a-4e39-ada3-3fca0ad3e9f8 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.103282 Sep 08 13:46:14.790712 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 4/9] 10.222.0.25 () {64 vars in 1336 bytes} [Thu Sep 8 13:46:14 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:46:17.495765 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-70be2068-7a9a-4ac8-89ae-3062f068faba admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:46:17.496784 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-70be2068-7a9a-4ac8-89ae-3062f068faba admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001467 Sep 08 13:46:17.497193 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 6/10] 10.222.0.25 () {54 vars in 931 bytes} [Thu Sep 8 13:46:17 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:46:17.977446 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-5e79c477-e35a-4ae4-81b0-0d06a5730359 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 10, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d2", "vcpus": 1, "name": "ds1G", "rxtx_factor": 1.0, "ram": 1024, "os-flavor-access:is_public": true}} {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:18.048638 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-5e79c477-e35a-4ae4-81b0-0d06a5730359 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.543936 Sep 08 13:46:18.049451 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 5/11] 10.222.0.25 () {64 vars in 1283 bytes} [Thu Sep 8 13:46:17 2022] POST /compute/v2.1/flavors => generated 404 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:46:18.062860 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-f7c8b2fa-a972-4cf5-83d2-2920a501969a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:18.177276 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-f7c8b2fa-a972-4cf5-83d2-2920a501969a admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.118467 Sep 08 13:46:18.177874 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 7/12] 10.222.0.25 () {64 vars in 1336 bytes} [Thu Sep 8 13:46:18 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:46:20.899054 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-5e79c477-e35a-4ae4-81b0-0d06a5730359 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:46:20.900158 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-5e79c477-e35a-4ae4-81b0-0d06a5730359 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001465 Sep 08 13:46:20.901540 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 6/13] 10.222.0.25 () {54 vars in 931 bytes} [Thu Sep 8 13:46:20 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:46:21.362492 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-9189e1f7-b3b7-4003-bf42-dfb912e520a9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 2, "rxtx_factor": 1.0, "disk": 10, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds2G", "swap": 0, "id": "d3", "ram": 2048, "os-flavor-access:is_public": true}} {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:21.454967 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-9189e1f7-b3b7-4003-bf42-dfb912e520a9 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.546082 Sep 08 13:46:21.454967 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 8/14] 10.222.0.25 () {64 vars in 1283 bytes} [Thu Sep 8 13:46:20 2022] POST /compute/v2.1/flavors => generated 404 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:46:21.471506 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-074bd892-e525-4516-828e-331893eaffbb admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:21.554464 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-074bd892-e525-4516-828e-331893eaffbb admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.091415 Sep 08 13:46:21.555231 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 7/15] 10.222.0.25 () {64 vars in 1336 bytes} [Thu Sep 8 13:46:21 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:46:24.320115 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-9189e1f7-b3b7-4003-bf42-dfb912e520a9 admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:46:24.322453 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-9189e1f7-b3b7-4003-bf42-dfb912e520a9 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.003476 Sep 08 13:46:24.323953 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 9/16] 10.222.0.25 () {54 vars in 931 bytes} [Thu Sep 8 13:46:24 2022] GET /compute/v2.1 => generated 388 bytes in 6 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:46:24.813332 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-4f3769d3-025d-4700-84a2-1c4c3f55f108 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d4", "os-flavor-access:is_public": true, "name": "ds4G", "rxtx_factor": 1.0, "vcpus": 4, "disk": 20, "swap": 0, "ram": 4096, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:24.893113 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-4f3769d3-025d-4700-84a2-1c4c3f55f108 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.559370 Sep 08 13:46:24.893604 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 8/17] 10.222.0.25 () {64 vars in 1283 bytes} [Thu Sep 8 13:46:24 2022] POST /compute/v2.1/flavors => generated 404 bytes in 561 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:46:24.907898 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-b9cc8f00-e254-4469-896e-c00eb29f7c70 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:25.008974 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-b9cc8f00-e254-4469-896e-c00eb29f7c70 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.104745 Sep 08 13:46:25.009568 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 10/18] 10.222.0.25 () {64 vars in 1336 bytes} [Thu Sep 8 13:46:24 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:46:28.178528 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-4f3769d3-025d-4700-84a2-1c4c3f55f108 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:46:28.179551 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-4f3769d3-025d-4700-84a2-1c4c3f55f108 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001537 Sep 08 13:46:28.179958 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 9/19] 10.222.0.25 () {54 vars in 931 bytes} [Thu Sep 8 13:46:28 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:46:28.706431 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-c1c8dbbe-be51-44ce-9a04-8d5dee97c84c admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:46:28.732321 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-c1c8dbbe-be51-44ce-9a04-8d5dee97c84c admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.545694 Sep 08 13:46:28.732912 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 11/20] 10.222.0.25 () {60 vars in 1292 bytes} [Thu Sep 8 13:46:28 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:46:31.571217 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-4f3769d3-025d-4700-84a2-1c4c3f55f108 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:46:31.571995 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-4f3769d3-025d-4700-84a2-1c4c3f55f108 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001498 Sep 08 13:46:31.572371 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 10/21] 10.222.0.25 () {54 vars in 931 bytes} [Thu Sep 8 13:46:31 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:46:32.003948 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-d386cbe5-26c8-47c3-a690-39039dd2641f admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.tiny", "rxtx_factor": 1.0, "disk": 1, "os-flavor-access:is_public": true, "id": "1", "ram": 512, "swap": 0, "vcpus": 1}} {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:32.076412 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-d386cbe5-26c8-47c3-a690-39039dd2641f admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.497135 Sep 08 13:46:32.076969 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 12/22] 10.222.0.25 () {64 vars in 1283 bytes} [Thu Sep 8 13:46:31 2022] POST /compute/v2.1/flavors => generated 402 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:46:32.089857 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-17873a86-9e01-4d07-94cc-258c78356011 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:32.159583 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-17873a86-9e01-4d07-94cc-258c78356011 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.073566 Sep 08 13:46:32.160178 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 11/23] 10.222.0.25 () {64 vars in 1333 bytes} [Thu Sep 8 13:46:32 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:46:34.790363 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-d386cbe5-26c8-47c3-a690-39039dd2641f admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:46:34.792448 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-d386cbe5-26c8-47c3-a690-39039dd2641f admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.003131 Sep 08 13:46:34.806563 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 13/24] 10.222.0.25 () {54 vars in 931 bytes} [Thu Sep 8 13:46:34 2022] GET /compute/v2.1 => generated 388 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:46:35.250367 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-1c7a58b7-9ea5-4bab-94aa-3647cc3c0ff2 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 20, "vcpus": 1, "id": "2", "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "ram": 2048, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0, "name": "m1.small"}} {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:35.340205 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-1c7a58b7-9ea5-4bab-94aa-3647cc3c0ff2 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.538000 Sep 08 13:46:35.341056 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 12/25] 10.222.0.25 () {64 vars in 1283 bytes} [Thu Sep 8 13:46:34 2022] POST /compute/v2.1/flavors => generated 405 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:46:35.359667 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-09b2a0ee-1e04-4c22-8a29-71204de5306a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:35.458205 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-09b2a0ee-1e04-4c22-8a29-71204de5306a admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.107330 Sep 08 13:46:35.458866 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 14/26] 10.222.0.25 () {64 vars in 1333 bytes} [Thu Sep 8 13:46:35 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:46:38.218507 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-1c7a58b7-9ea5-4bab-94aa-3647cc3c0ff2 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:46:38.219523 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-1c7a58b7-9ea5-4bab-94aa-3647cc3c0ff2 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001523 Sep 08 13:46:38.223281 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 13/27] 10.222.0.25 () {54 vars in 931 bytes} [Thu Sep 8 13:46:38 2022] GET /compute/v2.1 => generated 388 bytes in 6 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:46:38.769511 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-361a506d-c030-4cc9-af1d-5cd70060a4c7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "disk": 40, "id": "3", "name": "m1.medium", "os-flavor-access:is_public": true, "vcpus": 2, "swap": 0, "ram": 4096, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:38.881671 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-361a506d-c030-4cc9-af1d-5cd70060a4c7 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.655393 Sep 08 13:46:38.882712 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 15/28] 10.222.0.25 () {64 vars in 1283 bytes} [Thu Sep 8 13:46:38 2022] POST /compute/v2.1/flavors => generated 406 bytes in 657 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:46:38.894783 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-2505ddcc-ea66-4d3e-a774-e5101abd0cf0 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:38.900269 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG dbcounter [-] [112547] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=7,nova_api:SAVEPOINT=3,nova_api:RELEASE=3 {{(pid=112547) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:39.033190 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-2505ddcc-ea66-4d3e-a774-e5101abd0cf0 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.141778 Sep 08 13:46:39.048438 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 14/29] 10.222.0.25 () {64 vars in 1333 bytes} [Thu Sep 8 13:46:38 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:46:41.920161 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-361a506d-c030-4cc9-af1d-5cd70060a4c7 admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:46:41.921845 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-361a506d-c030-4cc9-af1d-5cd70060a4c7 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001930 Sep 08 13:46:41.922293 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 16/30] 10.222.0.25 () {54 vars in 931 bytes} [Thu Sep 8 13:46:41 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:46:42.463356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-b88ebb48-14db-4b8a-a410-8e4902e2d215 admin admin] Action: 'create', calling method: >, body: {"flavor": {"disk": 80, "id": "4", "os-flavor-access:is_public": true, "ram": 8192, "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "vcpus": 4, "name": "m1.large"}} {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:42.539811 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-b88ebb48-14db-4b8a-a410-8e4902e2d215 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.611333 Sep 08 13:46:42.541074 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 15/31] 10.222.0.25 () {64 vars in 1283 bytes} [Thu Sep 8 13:46:41 2022] POST /compute/v2.1/flavors => generated 405 bytes in 613 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:46:42.552851 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-9d496129-5c34-40b6-a201-a1d993f69be8 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:42.680236 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-9d496129-5c34-40b6-a201-a1d993f69be8 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.130969 Sep 08 13:46:42.689452 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 17/32] 10.222.0.25 () {64 vars in 1333 bytes} [Thu Sep 8 13:46:42 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:46:45.724631 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-b88ebb48-14db-4b8a-a410-8e4902e2d215 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:46:45.725939 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-b88ebb48-14db-4b8a-a410-8e4902e2d215 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001841 Sep 08 13:46:45.730556 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 16/33] 10.222.0.25 () {54 vars in 931 bytes} [Thu Sep 8 13:46:45 2022] GET /compute/v2.1 => generated 388 bytes in 7 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:46:46.279066 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-27fd259f-56cc-4553-b19c-7b58b41b6074 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "5", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 8, "name": "m1.xlarge", "rxtx_factor": 1.0, "ram": 16384, "os-flavor-access:is_public": true, "disk": 160}} {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:46.287383 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG dbcounter [-] [112548] Writing DB stats nova_api:SELECT=45,nova_api:SAVEPOINT=5,nova_api:INSERT=9,nova_api:RELEASE=5 {{(pid=112548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:46:46.387760 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-27fd259f-56cc-4553-b19c-7b58b41b6074 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.653802 Sep 08 13:46:46.388452 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 18/34] 10.222.0.25 () {64 vars in 1283 bytes} [Thu Sep 8 13:46:45 2022] POST /compute/v2.1/flavors => generated 408 bytes in 655 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:46:46.401400 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-f8414376-7453-4dd4-831d-5760999ab8d9 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:46:46.520755 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-f8414376-7453-4dd4-831d-5760999ab8d9 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.122760 Sep 08 13:46:46.521559 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 17/35] 10.222.0.25 () {64 vars in 1333 bytes} [Thu Sep 8 13:46:46 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:47:02.753089 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-27fd259f-56cc-4553-b19c-7b58b41b6074 admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:02.754099 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-27fd259f-56cc-4553-b19c-7b58b41b6074 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001455 Sep 08 13:47:02.754567 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 19/36] 10.222.0.25 () {54 vars in 931 bytes} [Thu Sep 8 13:47:02 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:47:02.765234 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG dbcounter [-] [112547] Writing DB stats nova_api:SELECT=14,nova_api:SAVEPOINT=2,nova_api:INSERT=3,nova_api:RELEASE=2 {{(pid=112547) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:47:03.167133 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-2df9c406-e364-4fa8-aa20-8bccc1e560e2 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:03.186380 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-2df9c406-e364-4fa8-aa20-8bccc1e560e2 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.425334 Sep 08 13:47:03.186888 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 18/37] 10.222.0.25 () {60 vars in 1292 bytes} [Thu Sep 8 13:47:02 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:47:04.514290 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-27fd259f-56cc-4553-b19c-7b58b41b6074 admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:04.515881 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-27fd259f-56cc-4553-b19c-7b58b41b6074 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002045 Sep 08 13:47:04.516320 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 20/38] 10.222.0.25 () {56 vars in 1074 bytes} [Thu Sep 8 13:47:04 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:47:05.043089 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-23ec7cc5-30da-48c6-a5a1-117058b1e1aa admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:05.049123 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.wsgi [None req-23ec7cc5-30da-48c6-a5a1-117058b1e1aa admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-23ec7cc5-30da-48c6-a5a1-117058b1e1aa admin admin] Request method failure captured: Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Accept: application/json Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Accept-Encoding: gzip, deflate Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Connection: keep-alive Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Content-Length: 0 Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Host: 10.222.0.25 Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Openstack-Api-Version: compute 2.92 Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Openstack-System-Scope: None Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: User-Agent: python-novaclient Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Auth-Token: gAAAAABjGfJY-jeh8eOefQsfSgf-SIYSo22MhgGrQeH98Q8xQtIcIFsn3cXZML0D-AIv_hV5pufq3Sbo5J4vM0pEpSOnU-Lg6pFphnfKVH7rRiHgSuojTBolbSILp9blDfQU0uPwI3Ds-r0J7yJ4RwTtbiYxP83VxC5NPVhFQLUKIPAhOHWGJoI Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Domain-Id: None Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Domain-Name: None Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Identity-Status: Confirmed Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Is-Admin-Project: True Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Openstack-Nova-Api-Version: 2.92 Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Project-Domain-Id: default Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Project-Domain-Name: Default Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Project-Id: 273637bb45fb4aad868a7d2db9ada393 Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Project-Name: admin Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Role: admin,member,reader Sep 08 13:47:05.053567 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Roles: admin,member,reader Sep 08 13:47:05.055254 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:47:05.055254 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Tenant: admin Sep 08 13:47:05.055254 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Tenant-Id: 273637bb45fb4aad868a7d2db9ada393 Sep 08 13:47:05.055254 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Tenant-Name: admin Sep 08 13:47:05.055254 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-User: admin Sep 08 13:47:05.055254 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-User-Domain-Id: default Sep 08 13:47:05.055254 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-User-Domain-Name: Default Sep 08 13:47:05.055254 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-User-Id: 4c5c0285558342f3b0682eed94f7ff4b Sep 08 13:47:05.055254 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-User-Name: admin Sep 08 13:47:05.055254 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: method: > Sep 08 13:47:05.055254 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: exception: Flavor 42 could not be found. Sep 08 13:47:05.055254 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: action_args: {'id': '42'} Sep 08 13:47:05.055254 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:47:05.055254 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:47:05.056198 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:47:05.057495 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:47:05.057495 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:47:05.057495 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:47:05.057495 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:47:05.057495 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 08 13:47:05.057495 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 08 13:47:05.057495 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 08 13:47:05.057495 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Sep 08 13:47:05.057495 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:47:05.057495 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Sep 08 13:47:05.057495 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:47:05.057495 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:47:05.057495 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:47:05.057495 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:47:05.057495 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:47:05.057495 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 08 13:47:05.057495 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Sep 08 13:47:05.057495 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-23ec7cc5-30da-48c6-a5a1-117058b1e1aa admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=112547) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:47:05.058680 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-23ec7cc5-30da-48c6-a5a1-117058b1e1aa admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.401250 Sep 08 13:47:05.058680 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 19/39] 10.222.0.25 () {60 vars in 1190 bytes} [Thu Sep 8 13:47:04 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 403 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:05.065665 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-f46dc70d-63b5-4db4-8530-53102a553462 admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:05.088394 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-f46dc70d-63b5-4db4-8530-53102a553462 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.026601 Sep 08 13:47:05.089182 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 21/40] 10.222.0.25 () {60 vars in 1210 bytes} [Thu Sep 8 13:47:05 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:47:05.102200 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-65ded073-7d1e-4a38-ab53-3aff6d3ef6f4 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:05.123334 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-65ded073-7d1e-4a38-ab53-3aff6d3ef6f4 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.027308 Sep 08 13:47:05.124498 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 20/41] 10.222.0.25 () {60 vars in 1210 bytes} [Thu Sep 8 13:47:05 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:47:05.134185 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-94149b77-4366-48c5-8d97-7812021d7165 admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:05.140130 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.wsgi [None req-94149b77-4366-48c5-8d97-7812021d7165 admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-94149b77-4366-48c5-8d97-7812021d7165 admin admin] Request method failure captured: Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Accept: application/json Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Accept-Encoding: gzip, deflate Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Connection: keep-alive Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Content-Length: 0 Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Host: 10.222.0.25 Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Openstack-Api-Version: compute 2.92 Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Openstack-System-Scope: None Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: User-Agent: python-novaclient Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Auth-Token: gAAAAABjGfJY-jeh8eOefQsfSgf-SIYSo22MhgGrQeH98Q8xQtIcIFsn3cXZML0D-AIv_hV5pufq3Sbo5J4vM0pEpSOnU-Lg6pFphnfKVH7rRiHgSuojTBolbSILp9blDfQU0uPwI3Ds-r0J7yJ4RwTtbiYxP83VxC5NPVhFQLUKIPAhOHWGJoI Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Domain-Id: None Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Domain-Name: None Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Identity-Status: Confirmed Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Is-Admin-Project: True Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Openstack-Nova-Api-Version: 2.92 Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Project-Domain-Id: default Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Project-Domain-Name: Default Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Project-Id: 273637bb45fb4aad868a7d2db9ada393 Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Project-Name: admin Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Role: admin,member,reader Sep 08 13:47:05.143347 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Roles: admin,member,reader Sep 08 13:47:05.145024 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:47:05.145024 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Tenant: admin Sep 08 13:47:05.145024 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Tenant-Id: 273637bb45fb4aad868a7d2db9ada393 Sep 08 13:47:05.145024 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Tenant-Name: admin Sep 08 13:47:05.145024 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-User: admin Sep 08 13:47:05.145024 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-User-Domain-Id: default Sep 08 13:47:05.145024 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-User-Domain-Name: Default Sep 08 13:47:05.145024 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-User-Id: 4c5c0285558342f3b0682eed94f7ff4b Sep 08 13:47:05.145024 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-User-Name: admin Sep 08 13:47:05.145024 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: method: > Sep 08 13:47:05.145024 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: exception: Flavor 42 could not be found. Sep 08 13:47:05.145024 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: action_args: {'id': '42'} Sep 08 13:47:05.145024 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:47:05.145024 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:47:05.146061 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:47:05.152868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:47:05.152868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:47:05.152868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:47:05.152868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:47:05.152868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 08 13:47:05.152868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 08 13:47:05.152868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 08 13:47:05.152868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Sep 08 13:47:05.152868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:47:05.152868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Sep 08 13:47:05.152868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:47:05.152868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:47:05.152868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:47:05.152868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:47:05.152868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:47:05.152868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 08 13:47:05.152868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Sep 08 13:47:05.152868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-94149b77-4366-48c5-8d97-7812021d7165 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=112548) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:47:05.153885 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-94149b77-4366-48c5-8d97-7812021d7165 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.014270 Sep 08 13:47:05.153885 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 22/42] 10.222.0.25 () {60 vars in 1190 bytes} [Thu Sep 8 13:47:05 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:06.388775 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-65ded073-7d1e-4a38-ab53-3aff6d3ef6f4 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:06.389849 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-65ded073-7d1e-4a38-ab53-3aff6d3ef6f4 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001485 Sep 08 13:47:06.390279 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 21/43] 10.222.0.25 () {56 vars in 1074 bytes} [Thu Sep 8 13:47:06 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:47:06.918993 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-88286702-61f7-4c4e-b746-7a02a8aee7eb admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:06.924043 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.wsgi [None req-88286702-61f7-4c4e-b746-7a02a8aee7eb admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-88286702-61f7-4c4e-b746-7a02a8aee7eb admin admin] Request method failure captured: Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Accept: application/json Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Accept-Encoding: gzip, deflate Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Connection: keep-alive Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Content-Length: 0 Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Host: 10.222.0.25 Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Openstack-Api-Version: compute 2.92 Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Openstack-System-Scope: None Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: User-Agent: python-novaclient Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Auth-Token: gAAAAABjGfJaZaLFeVDTOI72Ws9io31JHSScnlMsd3piclRgtlVc56E9MZPzmZsqXTMjXCPAviZBWXA-XcsKN3mTp__htQVJlGDr6zT97CtHHsJsOMq3Ci4m-39uB9Qzo3K7zLF5VTAE8mkSgwio_z-fBUS7gT_SkXrFltT7xEdGxIaM1EjHE-I Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Domain-Id: None Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Domain-Name: None Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Identity-Status: Confirmed Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Is-Admin-Project: True Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Openstack-Nova-Api-Version: 2.92 Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Project-Domain-Id: default Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Project-Domain-Name: Default Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Project-Id: 273637bb45fb4aad868a7d2db9ada393 Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Project-Name: admin Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Role: admin,member,reader Sep 08 13:47:06.925609 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Roles: admin,member,reader Sep 08 13:47:06.927203 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:47:06.927203 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Tenant: admin Sep 08 13:47:06.927203 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Tenant-Id: 273637bb45fb4aad868a7d2db9ada393 Sep 08 13:47:06.927203 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Tenant-Name: admin Sep 08 13:47:06.927203 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-User: admin Sep 08 13:47:06.927203 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-User-Domain-Id: default Sep 08 13:47:06.927203 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-User-Domain-Name: Default Sep 08 13:47:06.927203 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-User-Id: 4c5c0285558342f3b0682eed94f7ff4b Sep 08 13:47:06.927203 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-User-Name: admin Sep 08 13:47:06.927203 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: method: > Sep 08 13:47:06.927203 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: exception: Flavor 84 could not be found. Sep 08 13:47:06.927203 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: action_args: {'id': '84'} Sep 08 13:47:06.927203 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:47:06.927203 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:47:06.928184 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:47:06.929497 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:47:06.929497 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:47:06.929497 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:47:06.929497 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:47:06.929497 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 08 13:47:06.929497 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 08 13:47:06.929497 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 08 13:47:06.929497 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Sep 08 13:47:06.929497 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:47:06.929497 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Sep 08 13:47:06.929497 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:47:06.929497 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:47:06.929497 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:47:06.929497 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:47:06.929497 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:47:06.929497 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 08 13:47:06.929497 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Sep 08 13:47:06.929497 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-88286702-61f7-4c4e-b746-7a02a8aee7eb admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=112548) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:47:06.930857 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-88286702-61f7-4c4e-b746-7a02a8aee7eb admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.399446 Sep 08 13:47:06.930857 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 23/44] 10.222.0.25 () {60 vars in 1190 bytes} [Thu Sep 8 13:47:06 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 400 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:06.937521 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-ae278eef-dd17-4036-9e55-8590e211fbe0 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:06.952915 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-ae278eef-dd17-4036-9e55-8590e211fbe0 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018832 Sep 08 13:47:06.953456 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 22/45] 10.222.0.25 () {60 vars in 1210 bytes} [Thu Sep 8 13:47:06 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:47:06.963054 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-a6984194-628d-4492-9565-1ee080926384 admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:06.978666 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-a6984194-628d-4492-9565-1ee080926384 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019158 Sep 08 13:47:06.979197 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 24/46] 10.222.0.25 () {60 vars in 1210 bytes} [Thu Sep 8 13:47:06 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:47:06.988732 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-48894c4c-e6ec-4adb-8e8a-ece334ebefbe admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:06.994020 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.wsgi [None req-48894c4c-e6ec-4adb-8e8a-ece334ebefbe admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-48894c4c-e6ec-4adb-8e8a-ece334ebefbe admin admin] Request method failure captured: Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Accept: application/json Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Accept-Encoding: gzip, deflate Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Connection: keep-alive Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Content-Length: 0 Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Host: 10.222.0.25 Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Openstack-Api-Version: compute 2.92 Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Openstack-System-Scope: None Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: User-Agent: python-novaclient Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Auth-Token: gAAAAABjGfJaZaLFeVDTOI72Ws9io31JHSScnlMsd3piclRgtlVc56E9MZPzmZsqXTMjXCPAviZBWXA-XcsKN3mTp__htQVJlGDr6zT97CtHHsJsOMq3Ci4m-39uB9Qzo3K7zLF5VTAE8mkSgwio_z-fBUS7gT_SkXrFltT7xEdGxIaM1EjHE-I Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Domain-Id: None Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Domain-Name: None Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Identity-Status: Confirmed Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Is-Admin-Project: True Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Openstack-Nova-Api-Version: 2.92 Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Project-Domain-Id: default Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Project-Domain-Name: Default Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Project-Id: 273637bb45fb4aad868a7d2db9ada393 Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Project-Name: admin Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Role: admin,member,reader Sep 08 13:47:06.995580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Roles: admin,member,reader Sep 08 13:47:06.997189 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:47:06.997189 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Tenant: admin Sep 08 13:47:06.997189 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Tenant-Id: 273637bb45fb4aad868a7d2db9ada393 Sep 08 13:47:06.997189 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Tenant-Name: admin Sep 08 13:47:06.997189 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-User: admin Sep 08 13:47:06.997189 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-User-Domain-Id: default Sep 08 13:47:06.997189 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-User-Domain-Name: Default Sep 08 13:47:06.997189 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-User-Id: 4c5c0285558342f3b0682eed94f7ff4b Sep 08 13:47:06.997189 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-User-Name: admin Sep 08 13:47:06.997189 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: method: > Sep 08 13:47:06.997189 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: exception: Flavor 84 could not be found. Sep 08 13:47:06.997189 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: action_args: {'id': '84'} Sep 08 13:47:06.997189 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:47:06.997189 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:47:06.998130 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:47:07.000356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:47:07.000356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:47:07.000356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:47:07.000356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:47:07.000356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 08 13:47:07.000356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 08 13:47:07.000356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 08 13:47:07.000356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Sep 08 13:47:07.000356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:47:07.000356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Sep 08 13:47:07.000356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:47:07.000356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:47:07.000356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:47:07.000356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:47:07.000356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:47:07.000356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 08 13:47:07.000356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Sep 08 13:47:07.000356 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-48894c4c-e6ec-4adb-8e8a-ece334ebefbe admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=112547) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:47:07.001614 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-48894c4c-e6ec-4adb-8e8a-ece334ebefbe admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.011791 Sep 08 13:47:07.001614 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 23/47] 10.222.0.25 () {60 vars in 1190 bytes} [Thu Sep 8 13:47:06 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:08.339003 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-a6984194-628d-4492-9565-1ee080926384 admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:08.340067 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-a6984194-628d-4492-9565-1ee080926384 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001506 Sep 08 13:47:08.340460 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 25/48] 10.222.0.25 () {56 vars in 1074 bytes} [Thu Sep 8 13:47:08 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:47:08.869843 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-0d2d5da6-e6ab-46c4-b9f1-a629575646b7 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:08.875703 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.wsgi [None req-0d2d5da6-e6ab-46c4-b9f1-a629575646b7 admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-0d2d5da6-e6ab-46c4-b9f1-a629575646b7 admin admin] Request method failure captured: Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Accept: application/json Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Accept-Encoding: gzip, deflate Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Connection: keep-alive Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Content-Length: 0 Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Host: 10.222.0.25 Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Openstack-Api-Version: compute 2.92 Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Openstack-System-Scope: None Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: User-Agent: python-novaclient Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Auth-Token: gAAAAABjGfJcMZhfPZtfbCMlc9gdhepBf4cYKRaWNzVwYx71sKSlst-WUdMc0Ph-TrnrdJCH0wwzXqZdvQO_K6kSu7QiUgqCyLvZwQ5mdVMIEESxCX-uZEA8WEVsq2RpJqG3gAIbB4rPKjEYOXqThobgQ4M4h1ZdWEHMyHtll75xJ080ax5aLnk Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Domain-Id: None Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Domain-Name: None Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Identity-Status: Confirmed Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Is-Admin-Project: True Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Openstack-Nova-Api-Version: 2.92 Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Project-Domain-Id: default Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Project-Domain-Name: Default Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Project-Id: 273637bb45fb4aad868a7d2db9ada393 Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Project-Name: admin Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Role: admin,member,reader Sep 08 13:47:08.877181 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Roles: admin,member,reader Sep 08 13:47:08.878755 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:47:08.878755 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Tenant: admin Sep 08 13:47:08.878755 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Tenant-Id: 273637bb45fb4aad868a7d2db9ada393 Sep 08 13:47:08.878755 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-Tenant-Name: admin Sep 08 13:47:08.878755 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-User: admin Sep 08 13:47:08.878755 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-User-Domain-Id: default Sep 08 13:47:08.878755 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-User-Domain-Name: Default Sep 08 13:47:08.878755 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-User-Id: 4c5c0285558342f3b0682eed94f7ff4b Sep 08 13:47:08.878755 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: X-User-Name: admin Sep 08 13:47:08.878755 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: method: > Sep 08 13:47:08.878755 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: exception: Flavor 451 could not be found. Sep 08 13:47:08.878755 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: action_args: {'id': '451'} Sep 08 13:47:08.878755 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:47:08.878755 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:47:08.879822 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:47:08.881196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:47:08.881196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:47:08.881196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:47:08.881196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:47:08.881196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 08 13:47:08.881196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 08 13:47:08.881196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 08 13:47:08.881196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Sep 08 13:47:08.881196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:47:08.881196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Sep 08 13:47:08.881196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:47:08.881196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:47:08.881196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:47:08.881196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:47:08.881196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:47:08.881196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 08 13:47:08.881196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: ERROR nova.api.openstack.wsgi Sep 08 13:47:08.881196 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-0d2d5da6-e6ab-46c4-b9f1-a629575646b7 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=112547) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:47:08.882467 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-0d2d5da6-e6ab-46c4-b9f1-a629575646b7 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.399551 Sep 08 13:47:08.882665 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 24/49] 10.222.0.25 () {60 vars in 1193 bytes} [Thu Sep 8 13:47:08 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 404 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:08.888322 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-632ddff9-e1c6-4ee7-9dd2-1d88658c8fab admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:08.903923 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-632ddff9-e1c6-4ee7-9dd2-1d88658c8fab admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019188 Sep 08 13:47:08.904464 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 26/50] 10.222.0.25 () {60 vars in 1210 bytes} [Thu Sep 8 13:47:08 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:47:08.914712 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-3c7e4db5-3551-4ad7-9a6c-c82008567233 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:08.929626 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-3c7e4db5-3551-4ad7-9a6c-c82008567233 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.018465 Sep 08 13:47:08.930191 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 25/51] 10.222.0.25 () {60 vars in 1210 bytes} [Thu Sep 8 13:47:08 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:47:08.939868 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-316fb861-bb6e-402a-ba4c-d6bee091b844 admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:08.944864 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.wsgi [None req-316fb861-bb6e-402a-ba4c-d6bee091b844 admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-316fb861-bb6e-402a-ba4c-d6bee091b844 admin admin] Request method failure captured: Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Accept: application/json Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Accept-Encoding: gzip, deflate Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Connection: keep-alive Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Content-Length: 0 Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Host: 10.222.0.25 Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Openstack-Api-Version: compute 2.92 Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Openstack-System-Scope: None Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: User-Agent: python-novaclient Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Auth-Token: gAAAAABjGfJcMZhfPZtfbCMlc9gdhepBf4cYKRaWNzVwYx71sKSlst-WUdMc0Ph-TrnrdJCH0wwzXqZdvQO_K6kSu7QiUgqCyLvZwQ5mdVMIEESxCX-uZEA8WEVsq2RpJqG3gAIbB4rPKjEYOXqThobgQ4M4h1ZdWEHMyHtll75xJ080ax5aLnk Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Domain-Id: None Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Domain-Name: None Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Identity-Status: Confirmed Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Is-Admin-Project: True Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Openstack-Nova-Api-Version: 2.92 Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Project-Domain-Id: default Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Project-Domain-Name: Default Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Project-Id: 273637bb45fb4aad868a7d2db9ada393 Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Project-Name: admin Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Role: admin,member,reader Sep 08 13:47:08.946400 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Roles: admin,member,reader Sep 08 13:47:08.947882 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/273637bb45fb4aad868a7d2db9ada393"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:47:08.947882 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Tenant: admin Sep 08 13:47:08.947882 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Tenant-Id: 273637bb45fb4aad868a7d2db9ada393 Sep 08 13:47:08.947882 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-Tenant-Name: admin Sep 08 13:47:08.947882 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-User: admin Sep 08 13:47:08.947882 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-User-Domain-Id: default Sep 08 13:47:08.947882 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-User-Domain-Name: Default Sep 08 13:47:08.947882 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-User-Id: 4c5c0285558342f3b0682eed94f7ff4b Sep 08 13:47:08.947882 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: X-User-Name: admin Sep 08 13:47:08.947882 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: method: > Sep 08 13:47:08.947882 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: exception: Flavor 451 could not be found. Sep 08 13:47:08.947882 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: action_args: {'id': '451'} Sep 08 13:47:08.947882 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:47:08.947882 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:47:08.948771 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:47:08.949997 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:47:08.949997 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:47:08.949997 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:47:08.949997 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:47:08.949997 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 08 13:47:08.949997 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 08 13:47:08.949997 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 08 13:47:08.949997 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Sep 08 13:47:08.949997 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:47:08.949997 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Sep 08 13:47:08.949997 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:47:08.949997 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:47:08.949997 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:47:08.949997 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:47:08.949997 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:47:08.949997 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 08 13:47:08.949997 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: ERROR nova.api.openstack.wsgi Sep 08 13:47:08.949997 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-316fb861-bb6e-402a-ba4c-d6bee091b844 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=112548) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:47:08.951034 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-316fb861-bb6e-402a-ba4c-d6bee091b844 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.011766 Sep 08 13:47:08.951034 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 27/52] 10.222.0.25 () {60 vars in 1193 bytes} [Thu Sep 8 13:47:08 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 08 13:47:10.198803 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-3c7e4db5-3551-4ad7-9a6c-c82008567233 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:10.199820 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-3c7e4db5-3551-4ad7-9a6c-c82008567233 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001473 Sep 08 13:47:10.200216 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 26/53] 10.222.0.25 () {56 vars in 1074 bytes} [Thu Sep 8 13:47:10 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:47:10.735755 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-d7dac9c3-edc0-431e-9601-945d196ab6e3 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:47:10.830393 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-d7dac9c3-edc0-431e-9601-945d196ab6e3 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.492319 Sep 08 13:47:10.830963 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 28/54] 10.222.0.25 () {64 vars in 1235 bytes} [Thu Sep 8 13:47:10 2022] POST /compute/v2.1/flavors => generated 404 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:47:12.116670 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-3c7e4db5-3551-4ad7-9a6c-c82008567233 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:12.117708 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-3c7e4db5-3551-4ad7-9a6c-c82008567233 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001466 Sep 08 13:47:12.118111 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 27/55] 10.222.0.25 () {56 vars in 1074 bytes} [Thu Sep 8 13:47:12 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:47:12.655645 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-86714a3d-62fe-4edf-8fb5-f7cb57d787a7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:47:12.727489 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-86714a3d-62fe-4edf-8fb5-f7cb57d787a7 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.471530 Sep 08 13:47:12.728049 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 29/56] 10.222.0.25 () {64 vars in 1235 bytes} [Thu Sep 8 13:47:12 2022] POST /compute/v2.1/flavors => generated 405 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:47:14.029743 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-3c7e4db5-3551-4ad7-9a6c-c82008567233 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:14.030778 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-3c7e4db5-3551-4ad7-9a6c-c82008567233 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001502 Sep 08 13:47:14.031187 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 28/57] 10.222.0.25 () {56 vars in 1074 bytes} [Thu Sep 8 13:47:14 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:47:14.567750 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-985d90a9-7c8f-4cfb-be56-2b6bf3b45e26 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:47:14.664264 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-985d90a9-7c8f-4cfb-be56-2b6bf3b45e26 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.493412 Sep 08 13:47:14.664810 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 30/58] 10.222.0.25 () {64 vars in 1235 bytes} [Thu Sep 8 13:47:14 2022] POST /compute/v2.1/flavors => generated 407 bytes in 495 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:47:15.924563 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-3c7e4db5-3551-4ad7-9a6c-c82008567233 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:15.925558 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-3c7e4db5-3551-4ad7-9a6c-c82008567233 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001462 Sep 08 13:47:15.925953 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 29/59] 10.222.0.25 () {56 vars in 1074 bytes} [Thu Sep 8 13:47:15 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:47:16.474775 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-78352187-52a6-4426-a259-b17e8c4ee0a9 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:47:16.573012 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-78352187-52a6-4426-a259-b17e8c4ee0a9 admin admin] 10.222.0.25 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.498788 Sep 08 13:47:16.573489 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 31/60] 10.222.0.25 () {64 vars in 1235 bytes} [Thu Sep 8 13:47:16 2022] POST /compute/v2.1/flavors => generated 409 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:47:17.861867 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-3c7e4db5-3551-4ad7-9a6c-c82008567233 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:17.862914 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-3c7e4db5-3551-4ad7-9a6c-c82008567233 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001485 Sep 08 13:47:17.863325 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 30/61] 10.222.0.25 () {56 vars in 1074 bytes} [Thu Sep 8 13:47:17 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:47:18.401014 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-5410d7e2-0725-4c26-8c05-76bcd24559ba admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:47:18.421530 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-5410d7e2-0725-4c26-8c05-76bcd24559ba admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.410384 Sep 08 13:47:18.421991 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 32/62] 10.222.0.25 () {60 vars in 1202 bytes} [Thu Sep 8 13:47:18 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:48:16.224237 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-3c7e4db5-3551-4ad7-9a6c-c82008567233 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:48:16.225178 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-3c7e4db5-3551-4ad7-9a6c-c82008567233 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001485 Sep 08 13:48:16.225580 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 31/63] 10.222.0.25 () {54 vars in 931 bytes} [Thu Sep 8 13:48:16 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:48:16.236547 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG dbcounter [-] [112548] Writing DB stats nova_api:INSERT=5,nova_api:SELECT=28 {{(pid=112548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:16.792549 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-da415804-458f-452a-ac0f-a001fc375c79 admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:48:16.815276 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-da415804-458f-452a-ac0f-a001fc375c79 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.583117 Sep 08 13:48:16.815872 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 33/64] 10.222.0.25 () {60 vars in 1292 bytes} [Thu Sep 8 13:48:16 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 584 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:48:16.838149 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG dbcounter [-] [112547] Writing DB stats nova_api:SELECT=14 {{(pid=112547) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:48:16.842132 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-9e5dcf70-e866-4dfa-a192-1aedab3c2bb8 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:48:16.849943 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-9e5dcf70-e866-4dfa-a192-1aedab3c2bb8 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013382 Sep 08 13:48:16.857027 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 32/65] 10.222.0.25 () {60 vars in 1283 bytes} [Thu Sep 8 13:48:16 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 15 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:48:16.863865 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-ddd6faa7-977c-4de3-81a6-544243394bc6 admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:48:16.872150 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-ddd6faa7-977c-4de3-81a6-544243394bc6 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012665 Sep 08 13:48:16.872724 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 34/66] 10.222.0.25 () {60 vars in 1286 bytes} [Thu Sep 8 13:48:16 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:48:16.886366 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-7deaa74a-2af2-41cb-a0f6-37557ddad0b4 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:48:16.896927 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-7deaa74a-2af2-41cb-a0f6-37557ddad0b4 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.014685 Sep 08 13:48:16.897510 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 33/67] 10.222.0.25 () {60 vars in 1286 bytes} [Thu Sep 8 13:48:16 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 16 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:48:16.910254 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-e53b0589-43de-4a72-b2a3-655a13187980 admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:48:16.917988 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-e53b0589-43de-4a72-b2a3-655a13187980 admin admin] 10.222.0.25 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011524 Sep 08 13:48:16.918596 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 35/68] 10.222.0.25 () {60 vars in 1283 bytes} [Thu Sep 8 13:48:16 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:51:30.358332 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-7deaa74a-2af2-41cb-a0f6-37557ddad0b4 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:51:30.359337 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-7deaa74a-2af2-41cb-a0f6-37557ddad0b4 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001481 Sep 08 13:51:30.359735 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 34/69] 10.222.0.25 () {56 vars in 1074 bytes} [Thu Sep 8 13:51:30 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:51:30.501942 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG dbcounter [-] [112548] Writing DB stats nova_api:SELECT=6 {{(pid=112548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:30.663139 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-baa04131-e799-4e80-887b-cde08fddf2fb admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:51:30.667810 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-baa04131-e799-4e80-887b-cde08fddf2fb admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:51:30.668908 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-baa04131-e799-4e80-887b-cde08fddf2fb admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:51:30.669334 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-baa04131-e799-4e80-887b-cde08fddf2fb admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:51:30.670029 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-baa04131-e799-4e80-887b-cde08fddf2fb admin admin] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:51:30.670499 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-baa04131-e799-4e80-887b-cde08fddf2fb admin admin] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:51:30.670908 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-baa04131-e799-4e80-887b-cde08fddf2fb admin admin] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:51:30.701271 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-baa04131-e799-4e80-887b-cde08fddf2fb admin admin] 10.222.0.25 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.200683 Sep 08 13:51:30.701822 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 36/70] 10.222.0.25 () {60 vars in 1193 bytes} [Thu Sep 8 13:51:30 2022] GET /compute/v2.1/os-services => generated 791 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:51:31.804768 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-7deaa74a-2af2-41cb-a0f6-37557ddad0b4 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:51:31.805983 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-7deaa74a-2af2-41cb-a0f6-37557ddad0b4 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001481 Sep 08 13:51:31.805983 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 35/71] 10.222.0.25 () {56 vars in 1074 bytes} [Thu Sep 8 13:51:31 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:51:32.007730 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-eeaae382-6a1f-44ea-97eb-19fa548536ca admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:51:32.011270 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-eeaae382-6a1f-44ea-97eb-19fa548536ca admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:51:32.012332 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-eeaae382-6a1f-44ea-97eb-19fa548536ca admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:51:32.012759 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-eeaae382-6a1f-44ea-97eb-19fa548536ca admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:51:32.013473 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-eeaae382-6a1f-44ea-97eb-19fa548536ca admin admin] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:51:32.013918 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-eeaae382-6a1f-44ea-97eb-19fa548536ca admin admin] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:51:32.014360 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-eeaae382-6a1f-44ea-97eb-19fa548536ca admin admin] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:51:32.034127 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-eeaae382-6a1f-44ea-97eb-19fa548536ca admin admin] 10.222.0.25 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.086315 Sep 08 13:51:32.034725 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 37/72] 10.222.0.25 () {60 vars in 1193 bytes} [Thu Sep 8 13:51:31 2022] GET /compute/v2.1/os-services => generated 791 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:51:43.563175 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-7deaa74a-2af2-41cb-a0f6-37557ddad0b4 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:51:43.564168 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-7deaa74a-2af2-41cb-a0f6-37557ddad0b4 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001430 Sep 08 13:51:43.564556 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 36/73] 10.222.0.25 () {56 vars in 1074 bytes} [Thu Sep 8 13:51:43 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:51:43.712677 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG dbcounter [-] [112548] Writing DB stats nova_cell0:SELECT=4 {{(pid=112548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:43.713393 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG dbcounter [-] [112548] Writing DB stats nova_cell1:SELECT=4 {{(pid=112548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:43.714003 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG dbcounter [-] [112548] Writing DB stats nova_api:SELECT=8 {{(pid=112548) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:51:43.762692 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-33e26f19-d7c0-4024-b7a0-b1ca30b4ded6 admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:51:43.766102 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-33e26f19-d7c0-4024-b7a0-b1ca30b4ded6 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:51:43.766598 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-33e26f19-d7c0-4024-b7a0-b1ca30b4ded6 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:51:43.767021 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-33e26f19-d7c0-4024-b7a0-b1ca30b4ded6 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:51:43.767723 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-33e26f19-d7c0-4024-b7a0-b1ca30b4ded6 admin admin] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:51:43.768174 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-33e26f19-d7c0-4024-b7a0-b1ca30b4ded6 admin admin] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:51:43.768576 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-33e26f19-d7c0-4024-b7a0-b1ca30b4ded6 admin admin] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:51:43.794204 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-33e26f19-d7c0-4024-b7a0-b1ca30b4ded6 admin admin] 10.222.0.25 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.085498 Sep 08 13:51:43.794763 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 38/74] 10.222.0.25 () {60 vars in 1193 bytes} [Thu Sep 8 13:51:43 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:51:44.887866 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG nova.api.openstack.wsgi [None req-7deaa74a-2af2-41cb-a0f6-37557ddad0b4 admin admin] Calling method '>' {{(pid=112547) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:51:44.888865 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: INFO nova.api.openstack.requestlog [None req-7deaa74a-2af2-41cb-a0f6-37557ddad0b4 admin admin] 10.222.0.25 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002011 Sep 08 13:51:44.889260 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: [pid: 112547|app: 0|req: 37/75] 10.222.0.25 () {56 vars in 1074 bytes} [Thu Sep 8 13:51:44 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:51:45.081757 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG nova.api.openstack.wsgi [None req-3c87967d-6d4b-46f5-a5ca-93f9fcba803f admin admin] Calling method '>' {{(pid=112548) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:51:45.085143 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-3c87967d-6d4b-46f5-a5ca-93f9fcba803f admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:51:45.085618 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-3c87967d-6d4b-46f5-a5ca-93f9fcba803f admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:51:45.086044 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-3c87967d-6d4b-46f5-a5ca-93f9fcba803f admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:51:45.086784 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-3c87967d-6d4b-46f5-a5ca-93f9fcba803f admin admin] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:51:45.087228 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-3c87967d-6d4b-46f5-a5ca-93f9fcba803f admin admin] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:51:45.087645 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: DEBUG oslo_concurrency.lockutils [None req-3c87967d-6d4b-46f5-a5ca-93f9fcba803f admin admin] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=112548) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:51:45.110436 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: INFO nova.api.openstack.requestlog [None req-3c87967d-6d4b-46f5-a5ca-93f9fcba803f admin admin] 10.222.0.25 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.081658 Sep 08 13:51:45.111122 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: [pid: 112548|app: 0|req: 39/76] 10.222.0.25 () {60 vars in 1193 bytes} [Thu Sep 8 13:51:45 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:52:44.061899 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: Thu Sep 8 13:52:44 2022 - graceful shutdown triggered... Sep 08 13:52:44.061951 dv-cinder-856460-1-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Sep 08 13:52:44.063442 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: Gracefully killing worker 1 (pid: 112547)... Sep 08 13:52:44.064040 dv-cinder-856460-1-iscsi devstack@n-api.service[112548]: Gracefully killing worker 2 (pid: 112548)... Sep 08 13:52:44.190387 dv-cinder-856460-1-iscsi devstack@n-api.service[112547]: DEBUG dbcounter [-] [112547] Writing DB stats nova_api:SELECT=4 {{(pid=112547) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:45.064617 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: worker 1 buried after 1 seconds Sep 08 13:52:45.064617 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: worker 2 buried after 1 seconds Sep 08 13:52:45.064617 dv-cinder-856460-1-iscsi devstack@n-api.service[112546]: goodbye to uWSGI. Sep 08 13:52:45.066199 dv-cinder-856460-1-iscsi systemd[1]: devstack@n-api.service: Succeeded. Sep 08 13:52:45.066752 dv-cinder-856460-1-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Sep 08 13:52:45.069644 dv-cinder-856460-1-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 08 13:52:45.080302 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 08 13:52:45.082575 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 8 13:52:45 2022] *** Sep 08 13:52:45.082575 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 08 13:52:45.082575 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 08 13:52:45.082575 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: nodename: dv-cinder-856460-1-iscsi Sep 08 13:52:45.082575 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: machine: x86_64 Sep 08 13:52:45.082575 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: clock source: unix Sep 08 13:52:45.082575 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: pcre jit disabled Sep 08 13:52:45.082575 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: detected number of CPU cores: 4 Sep 08 13:52:45.082575 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: current working directory: / Sep 08 13:52:45.082575 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: detected binary path: /usr/bin/uwsgi-core Sep 08 13:52:45.082575 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: your processes number limit is 47680 Sep 08 13:52:45.082575 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: your memory page size is 4096 bytes Sep 08 13:52:45.082575 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: detected max file descriptor number: 2048 Sep 08 13:52:45.082575 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: lock engine: pthread robust mutexes Sep 08 13:52:45.083472 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: thunder lock: enabled Sep 08 13:52:45.083472 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 08 13:52:45.083472 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 08 13:52:45.123618 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: Python main interpreter initialized at 0x563d7f70b9c0 Sep 08 13:52:45.123618 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: python threads support enabled Sep 08 13:52:45.123618 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: your server socket listen backlog is limited to 100 connections Sep 08 13:52:45.123618 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: your mercy for graceful operations on workers is 90 seconds Sep 08 13:52:45.124144 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: mapped 403077 bytes (393 KB) for 2 cores Sep 08 13:52:45.124144 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: *** Operational MODE: preforking *** Sep 08 13:52:45.124363 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: *** uWSGI is running in multiple interpreter mode *** Sep 08 13:52:45.124363 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: spawned uWSGI master process (pid: 127586) Sep 08 13:52:45.124585 dv-cinder-856460-1-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 08 13:52:45.125176 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: spawned uWSGI worker 1 (pid: 127601, cores: 1) Sep 08 13:52:45.125946 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: spawned uWSGI worker 2 (pid: 127602, cores: 1) Sep 08 13:52:45.125946 dv-cinder-856460-1-iscsi devstack@n-api.service[127586]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 08 13:52:45.804735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 08 13:52:45.808790 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 08 13:52:46.980348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:52:46.980348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: from cryptography.utils import int_from_bytes Sep 08 13:52:46.981079 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:52:46.981079 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: from cryptography.utils import int_from_bytes Sep 08 13:52:46.987852 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:52:46.987852 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: from cryptography.utils import int_from_bytes Sep 08 13:52:46.988450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:52:46.988450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: from cryptography.utils import int_from_bytes Sep 08 13:52:49.459991 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 08 13:52:49.460624 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 08 13:52:49.460723 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 08 13:52:49.461070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 08 13:52:49.461159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 08 13:52:49.461252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 08 13:52:49.461337 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 08 13:52:49.461417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 08 13:52:49.461711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 08 13:52:49.461828 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.461967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 08 13:52:49.462068 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.462364 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.462481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.462686 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.462801 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.462993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.463102 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.463304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.463410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.463619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.463743 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.463926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.464049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.464800 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.464920 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.465068 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.465171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.465380 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.465546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.465687 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.465822 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-856460-1-iscsi {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.465975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.466176 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.466458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-856460-1-iscsi {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.466565 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.466765 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.466868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.467077 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.467180 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.467378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.467476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.467692 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.467794 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.467991 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.468091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.468314 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.468506 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.469029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.469135 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.469320 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.469441 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.469609 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.469876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.470002 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.470235 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.470324 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.470548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.470673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.470853 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.470963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.471156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.471246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.471468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.471550 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.471793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.471890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.472103 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.472240 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.472511 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-856460-1-iscsi {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.472651 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.472889 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.473030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-856460-1-iscsi {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.473165 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.473352 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.473480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.473672 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.473839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.473988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.474183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.474432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.474573 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.474739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.474866 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.475046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.475170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.475364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.475502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.475686 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.475814 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.475993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.476123 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.476306 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.476434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.476622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.477854 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.477854 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.478236 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.479438 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.479620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.480057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.480227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.480384 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.480559 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.480835 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.481003 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.481154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.481307 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.481458 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.481629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.481782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.481956 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.482102 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.482282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.482427 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.482599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.483190 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.483366 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.483526 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.483526 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.483829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.483999 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.484155 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.484313 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.484484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.484484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.484872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.485030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.485175 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.485329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.485474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.485474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.485784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.485928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.486079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.486278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.486424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.486424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.486718 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.486870 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.487024 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.487174 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.487323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.487487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.487643 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.487798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.487947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.488098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.488242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.488392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.488544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.488689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.488843 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.25 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.488996 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.489145 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.25 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.489299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.489501 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.489658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.25 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.489813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.489963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.25 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.490280 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.490440 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.490595 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.490740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.491015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.491174 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.491327 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.491485 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.491637 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.491833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.491992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.492134 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.492289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.492442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.492593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.492735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.492987 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.493140 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.493371 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.493529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.493679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.493679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.493953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.494103 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.494248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.494248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.494660 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.494956 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.495104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.495243 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.495525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.495525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.495798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.495948 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.496088 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.496227 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.496390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.496484 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.496618 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.496838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.496975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.497115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.497266 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.497415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.497562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.497707 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.506605 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.506722 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.506841 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.506941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.507032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.507111 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.507184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.507273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.507600 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.507691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.507765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.507844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.507927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.508013 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.508088 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.508166 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.508247 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.508347 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.508444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.508530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.508622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.508696 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.508854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.508931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.509011 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.509100 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.509174 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.509252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.509334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.509406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.509477 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.509555 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.509636 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.509709 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.509795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.509875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.509955 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.510039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.510122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.510262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.510345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.510424 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.510506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.510579 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.510656 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.510742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.510825 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.510897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.510976 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.511776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.511883 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.511966 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.512055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.512055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.512204 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.512280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.512350 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.512428 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.512428 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.512556 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.512643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.512643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.512780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.512861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.512934 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.513014 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.513087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.513163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.513244 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.513323 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.513402 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.513481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.513561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.513640 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.513721 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.513828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.513915 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.513997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.514071 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.514167 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:52:49.514256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.514331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.514410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.514688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.514783 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.514867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.514947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.515020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.515103 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.515177 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.515248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.515248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.515248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.515248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.515516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.515595 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.515667 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.515750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.515832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.515926 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.516025 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.516118 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.516118 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.516281 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.516374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.516466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.516565 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.516660 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.516742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.516822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.516895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.516973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.517063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.517158 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.517255 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.517348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.517441 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.517534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.517632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.517720 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.517811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.517897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.517981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.518065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.518182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.518278 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.518365 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.518440 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.518510 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.518602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.518697 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.518790 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.518883 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.518979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.519073 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.519165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.519259 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.519259 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.519432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.519518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.519612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.519715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.519803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.519894 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.519991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.520084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.520170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.520269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.520356 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.520447 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.520534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.520626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.520719 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.520812 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.520898 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.520988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.521082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.521175 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.521268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.521362 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.521455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.521548 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.521642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.521752 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.522023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.522129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.522327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.522416 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.522497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.522571 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.522652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.522725 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.522811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.522899 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.522899 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.523063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.523158 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.523252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.523338 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.523424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.523515 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.523603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.523691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.523779 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.523870 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.523965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.524039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.524039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.524249 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.524334 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.525036 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.525163 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.525255 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.525355 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.525451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.525534 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.525619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.525710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.525797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.525895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.526025 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.526122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.526371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.526466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.526760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.527410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.527588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.527679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.527896 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.528013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.528215 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.528367 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.528524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.528788 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.528908 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.529137 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.529240 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.529442 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.529687 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.529776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.529872 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.530047 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.530322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.530432 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.530620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.530714 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.530978 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.531112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.531287 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.531399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.531606 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.531780 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.531917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.532263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.532421 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.532562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.532708 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.532860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.533013 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.533166 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.533316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.533477 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.533626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.533766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.533914 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.534098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.534268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.534418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.534558 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.534718 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.534861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.535011 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.535154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.535299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.535441 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.535586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.535724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.535883 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.536022 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.536168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.536307 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.536445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.536591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.536741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.536884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.537031 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.537167 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.537315 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.537468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.537610 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.537765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.537911 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.538060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.538267 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.538423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.538560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.538697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.538784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.538921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.539058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.539209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.539357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.539495 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.539676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.539807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.540011 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.540114 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.540310 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.540408 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.540613 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.540713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.540925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.541018 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.541229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.541321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.541532 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.541633 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.541836 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.541940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.542158 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.542266 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.542502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.542621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.542820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.542920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.543127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.543221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.543442 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.543544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.543749 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.543851 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.544070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.544173 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.544390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.544591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.544705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.544799 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.544994 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.545134 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.545298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.545406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.545607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.545700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.545916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.546015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.546266 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.550991 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.551106 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.551203 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.551203 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.551358 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.551459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.551547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.551631 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.551722 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.551851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.551976 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.552079 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.552161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.552240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.552326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.552409 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.552503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.552603 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.552697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.552784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.552857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.552937 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.553009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.553094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.553169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.553248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.553327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.553400 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.553478 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.553564 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.553645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.553736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.553823 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.553921 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.554017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.554092 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.554216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.554295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.554368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.554446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.554528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.554603 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.554603 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.554751 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.554824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.554949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.555124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.555124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.555124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.555124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.555387 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.555460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.555545 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.555644 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.555724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.555803 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.556029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.556123 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.556199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.556278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.556358 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.556438 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.556511 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.556591 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.556664 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.556742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.556847 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.556935 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.557233 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.557363 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.557632 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.557994 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.558378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.558475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.558553 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.558784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.558888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.559093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.559195 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.559402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.559670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.559759 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.559842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.560052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.560155 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.560354 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.560462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.560661 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.560782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.560968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.561107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.561315 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.561422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.561626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.561719 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.561928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.562018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.562263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.562378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.562585 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.562693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.562896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.563001 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.563209 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.563311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.563519 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.563613 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.563829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.564007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.564173 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.564280 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.564475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.564617 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.564765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.564907 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.565056 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.565204 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.565344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.565492 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.565632 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.565779 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.565929 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.566087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.566270 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.566403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.566556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.566705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.566857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.567020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.567173 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.567323 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.567515 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.567680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.567835 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.567980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.568124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.568278 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.568421 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.568572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.568725 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.568900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.569052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.569198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.569342 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.569499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.569644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.569813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.569972 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.570125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.570335 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.570482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.570644 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.570790 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.570968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.571208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.571368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.571511 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.571663 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.571809 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.571979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.572126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.572269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.572426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.572574 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.572728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.572882 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.572882 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.573114 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.573190 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.573463 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.574295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.574408 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.574506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.574600 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.574813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.574915 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.575169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.575272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.575477 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.575598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.575792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.575888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.576094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.576233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.577616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.577776 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.577921 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.578060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.578060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.585489 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.585631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.585764 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.585889 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.586010 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.586010 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.586286 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.586401 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.586527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.586527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.586753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.586872 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.586983 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.587103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.587221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.587350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.587466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.587584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.587743 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.587847 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.587988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.588202 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.588411 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.588570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.588728 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.588892 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.589038 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.589292 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.589951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.590099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.590301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.590463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.590619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.590765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.590910 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.591070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.591244 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.591407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.591681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.591681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.591965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.592110 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.592346 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.592503 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.592657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.592802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.592949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.593100 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.593375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.593547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.593695 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.593839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.593984 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.594128 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.594389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.594561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.594833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.594993 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.595139 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.595299 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.595452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.595604 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.595761 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.595905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.596057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.596212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.596357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.596501 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.596653 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.596806 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.596977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.596977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.596977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.597434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.597599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.597599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.597599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.597599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.597599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.597599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.597599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.597599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.598758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.598758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.598758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.598758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.598758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.598758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.598758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.598758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.599934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.599934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.599934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.599934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.599934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.600628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.600773 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.600915 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.601059 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.601059 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.601363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.601515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.601515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.601917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.602068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.602068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.602068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.602517 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.602662 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.602814 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.602967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.602967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.603256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.603509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.603662 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.603807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.603807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.604097 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.604253 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.604407 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.604553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.604696 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.605029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.605193 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.605339 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.605482 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.605648 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.605648 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.605933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.606079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.606265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.606422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.606566 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.606711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.606864 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.607016 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.607167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.607312 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.607484 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.607638 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.607792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.607936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.608118 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.608329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.608542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.608783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.609009 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.609210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.609358 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.609510 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.609654 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.609821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.610030 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.610198 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.610348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.610494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.610647 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.610807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.610959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.611112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.611264 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.611410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.611568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.611722 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.611878 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.612234 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.612234 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.612535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.612681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.612681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.613065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.613222 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.613374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.613544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.613544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.613915 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.614070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.614262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.614418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.614571 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.614725 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.614870 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.615016 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.615167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.615425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.615584 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.615816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.615976 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.616132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.616132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.616132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.616132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.616132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.616132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.616962 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.617158 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.617310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.617465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.617619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.617772 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.617916 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.618059 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.618249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.618408 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.618561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.618713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.618857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.619000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.619142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.619301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.619445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.619596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.619748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.619892 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.620035 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.620202 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.620355 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.620528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.620681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.620825 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.620975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.621161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.621353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.621585 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.621585 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.621585 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.621937 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.622052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.622189 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.622406 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.622406 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.622622 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.622622 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.629234 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.629436 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.629602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.629602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.629911 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.630072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.630302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.630452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.630610 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.630758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.630903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.631067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.631223 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.631384 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.631581 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.631732 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.631890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.632036 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.632182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.632336 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.632869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.633033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.633191 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.633338 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.633544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.633700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.633875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.634111 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.634519 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.634690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.634838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.634986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.635141 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.635312 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.635469 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.635733 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.635900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.636592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.636757 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.637155 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.637333 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.637518 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.637668 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.637815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.638017 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.638207 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.638207 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.638626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.638888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.639048 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.639205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.639360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.639515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.639661 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.639807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.639807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.640721 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.640875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.641033 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.641181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.641335 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.641551 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.641740 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.641899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.641899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.642264 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.642430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.642621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.642772 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.643118 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.643298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.643298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.643602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.643749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.643895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.644048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.644048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.644048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.644048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.644048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.644048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.644048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.644048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.645867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.645867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.645867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.645867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.645867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.645867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.645867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.645867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.647125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.647125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.647125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.647125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.647125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.647125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.648024 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.648236 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.648236 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.648527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.648777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.649062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.649225 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.649379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.649534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.649697 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.649858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.650013 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.650237 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.650425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.650655 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.650655 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.650969 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.651152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.651304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.651451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.651605 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.652339 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.652492 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.652655 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.652813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.652970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.653150 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.653432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.653584 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.653739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.653739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.654031 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.654216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.654461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.654615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.654762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.654911 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.655067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.655230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.655444 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.655655 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.655813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.655962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.656144 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.656390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.656544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.656692 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.656692 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.656968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.657131 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.657295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.657486 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.657486 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.657785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.657932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.658088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.658284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.658446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.658611 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.658611 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.659079 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.659350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.659547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.659711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.659711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.659998 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.660156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.660310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.660466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.660620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.660768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.660768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.661072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.661221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.661367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.661524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.661670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.661825 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.662065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.662287 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.662447 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.662603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.662825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.662974 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.663136 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.663291 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.663600 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.663755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.663915 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.664077 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.664225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.664383 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.664646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.664801 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.664948 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.665110 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.665257 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.665413 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.665574 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.665729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.665883 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.666132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.666322 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.666620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.666817 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.666978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.667134 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.667378 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.667539 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.667720 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.667872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.668021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.668178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.668395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.668395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.668697 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.668844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.669007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.669269 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.669269 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.669686 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.669841 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.669988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.670152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.670333 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.670501 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.670650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.670804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.670959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.671120 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.671273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.671420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.671573 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.671766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.671918 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.672074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.672220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.672377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.672531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.672679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.672835 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.672990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.673145 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.673300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.673466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.673749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.673919 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.674068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.674306 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.674456 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.674641 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.674902 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.675069 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.675218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.675218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.675496 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.675643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.675842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.675842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.676150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.676150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.676436 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.676436 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.676921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.677083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.677230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.677384 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.677384 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.677676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.677676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.677972 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.678119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.678290 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.678480 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.678627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.683330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.683330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.683666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.683954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.683954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.684254 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.684403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.684538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.684678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.684811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.685075 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.685229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.685372 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.685511 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.685644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.685791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.685932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.686107 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.686275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.686421 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.686555 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.686687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.686818 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.687070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.687215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.687360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.687494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.687625 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.687804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.687804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.688078 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.688249 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.688384 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.688562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.688710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.688861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.689002 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.689161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.689300 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.689533 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.689698 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.689921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.690066 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.690249 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.690398 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.690614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.690845 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.690988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.691131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.691476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.691476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.691753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.691895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.692055 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.692190 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.692362 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.692506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.692652 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.692785 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.692923 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.693087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.693250 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.693397 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.693397 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.693666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.693798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.693943 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.694202 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.694342 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.694514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.694647 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.694782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.694782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.695046 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.695046 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.695403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.695403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.695670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.695670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.695924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.695924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.696186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.696318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.696462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.696595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.696595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.696931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.697087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.697087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.697634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.697858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.698029 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.698261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.698261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.699149 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.699322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.699495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.699712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.699712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.700126 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.700284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.700284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.700543 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.700790 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.701034 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.701178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.701312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.701643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.701643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.701918 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.702057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.702216 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.702349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.702349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.702633 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.702768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.702911 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.703044 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.703044 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.703044 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.703439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.703581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.703581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.703838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.703971 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.703971 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.704366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.704550 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.704686 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.704816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.704816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.705082 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.705238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.705238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.705640 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.705781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.705922 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.706069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.706069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.706350 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.706482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.706666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.706813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.706813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.706813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.707198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.707339 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.707339 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.707790 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.707926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.708057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.708217 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.708217 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.708478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.708620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.708620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.708620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.708620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.709154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.709323 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.709471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.709604 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.709604 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.709866 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.709997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.709997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.710410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.710545 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.710720 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.710945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.710945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.710945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.711456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.711765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.711765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.712023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.712156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.712156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.712886 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.713044 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.713187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.713321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.713321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.713638 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.713772 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.713772 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.714029 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.714223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.714223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.715118 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.715260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.715260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.715663 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.715816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.715816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.716088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.716230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.716364 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.716503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.716503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.716778 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.716919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.716919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.717178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.717317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.717317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.717590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.717756 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.717895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.718208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.718208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.718481 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.718617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.718617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.718911 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.719051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.719051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.719356 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.719491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.719630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.719797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.719797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.720063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.720224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.720438 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.720582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.720715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.720848 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.721077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.721306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.721306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.721614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.721765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.721765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.721765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.722443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.722577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.722708 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.722840 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.722840 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.723102 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.723235 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.723367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.723507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.723507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.723804 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.723953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.723953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.724203 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.724331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.724331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.724588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.724727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.724727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.725000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.725238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.725238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.725507 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.725727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.725871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.726005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.726005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.726311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.726479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.726479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.726711 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.726886 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.726886 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.727149 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.727368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.727511 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.727646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.727646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.727933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.728081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.728236 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.728474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.728474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.728805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.728939 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.729105 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.729249 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.729249 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.729560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.729728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.729728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.730054 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.730203 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.730350 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.730581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.730581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.730581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.730970 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.731205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.731430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.731566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.731566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.731566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.731566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.732282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.732416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.732556 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.732697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=127601) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 08 13:52:49.732844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.732844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.732844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.732844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.732844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.732844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.732844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.732844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.733848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.733986 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.734329 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.734714 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.735111 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.735497 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.735881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.736261 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.736641 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.737025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.737417 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.737796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.738199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.738593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.739010 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.739387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.739785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.740175 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.740623 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.740933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.741315 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.741694 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.742177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.742481 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.742885 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.743272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.743650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.744148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.744554 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.744942 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.745332 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.746278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.746673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.747070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.747447 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.747846 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.748227 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.748675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.749027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.749460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.749818 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.750238 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.750618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.750996 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.751379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.751768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.752206 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.752509 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.752881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.753261 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.753634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.754008 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.754401 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.754789 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.755216 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.755587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.755960 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.756336 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.756706 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.757078 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.757710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.758087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.758479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.758854 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.759222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.759595 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.760109 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.760396 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.760792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.761806 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.762196 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.762568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.762918 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.763299 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.763691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.764083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.764469 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.764849 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.765251 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.765643 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.766021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.766428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.766795 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.767174 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.767564 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.767956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.768349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.768724 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.769120 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.769509 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.769896 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.770325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.770704 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.771099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.771478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.771920 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.772229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.772675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.772986 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.773367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.773745 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.774123 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.774529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.774914 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.775330 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.775695 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.776070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.776703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.776861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.777274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.777650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.778062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.778476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.778858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.779795 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.779795 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:52:49.780075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=127602) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 08 13:52:49.792214 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO dbcounter [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] Registered counter for database nova_api Sep 08 13:52:49.814786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_db.sqlalchemy.engines [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127601) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:52:49.821962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writer thread running {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:52:49.843252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO dbcounter [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] Registered counter for database nova_api Sep 08 13:52:49.863626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_db.sqlalchemy.engines [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127602) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:52:49.870331 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writer thread running {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:52:49.879672 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.context [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf(cell1) {{(pid=127601) load_cells /opt/stack/nova/nova/context.py:464}} Sep 08 13:52:49.884041 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:52:49.884516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:52:49.887074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:52:49.887827 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:52:49.888271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:52:49.890294 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:52:49.892280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO dbcounter [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] Registered counter for database nova_cell0 Sep 08 13:52:49.896410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO dbcounter [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] Registered counter for database nova_cell1 Sep 08 13:52:49.903271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_db.sqlalchemy.engines [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127601) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:52:49.905475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_db.sqlalchemy.engines [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127601) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:52:49.911135 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writer thread running {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:52:49.912428 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writer thread running {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:52:49.927420 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.context [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf(cell1) {{(pid=127602) load_cells /opt/stack/nova/nova/context.py:464}} Sep 08 13:52:49.931480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:52:49.931979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:52:49.934364 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:52:49.934828 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:52:49.935274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:52:49.937260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:52:49.939293 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO dbcounter [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] Registered counter for database nova_cell0 Sep 08 13:52:49.943318 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO dbcounter [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] Registered counter for database nova_cell1 Sep 08 13:52:49.950803 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_db.sqlalchemy.engines [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127602) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:52:49.953004 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_db.sqlalchemy.engines [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127602) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:52:49.957999 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writer thread running {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:52:49.963291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writer thread running {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:52:50.059257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO dbcounter [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] Registered counter for database nova_cell0 Sep 08 13:52:50.065986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_db.sqlalchemy.engines [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127601) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:52:50.070520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writer thread running {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:52:50.105461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO dbcounter [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] Registered counter for database nova_cell0 Sep 08 13:52:50.112247 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_db.sqlalchemy.engines [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=127602) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:52:50.117027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writer thread running {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:52:50.396052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: WARNING keystonemiddleware.auth_token [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 08 13:52:50.403395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: WARNING keystonemiddleware.auth_token [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 08 13:52:50.436318 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: WARNING keystonemiddleware.auth_token [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 08 13:52:50.443660 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: WARNING keystonemiddleware.auth_token [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 08 13:52:50.756381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: WARNING keystonemiddleware.auth_token [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 08 13:52:50.762735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: WARNING keystonemiddleware.auth_token [None req-67e01b65-6d77-405b-9faf-0487bf340bbe None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 08 13:52:50.768228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563d7f70b9c0 pid: 127601 (default app) Sep 08 13:52:50.789697 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: WARNING keystonemiddleware.auth_token [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 08 13:52:50.796347 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: WARNING keystonemiddleware.auth_token [None req-1c84bb84-f127-4b88-9038-b9f3eab92fc2 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 08 13:52:50.801687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x563d7f70b9c0 pid: 127602 (default app) Sep 08 13:53:27.524844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=3 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:27.526483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=3 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:27.527974 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=3 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:27.528771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=3 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:27.986745 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=3 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:27.988514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=3 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:27.989270 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=3 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:27.989988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=3 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:28.780448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-af337801-861e-4751-8737-3465acb1ddeb tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Action: 'create', calling method: .version_select at 0x7fce030be3a0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-1298805441"}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:53:29.233675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b1bf7fe1-500c-44e1-978c-c162a3f7a032 tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] Action: 'create', calling method: .version_select at 0x7fce030be3a0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1595788840", "metadata": {"Type": "work"}}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:53:31.670303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-af337801-861e-4751-8737-3465acb1ddeb tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 200 len: 345 microversion: 2.1 time: 4.155209 Sep 08 13:53:31.671317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1/1] 10.222.0.25 () {60 vars in 1155 bytes} [Thu Sep 8 13:53:27 2022] POST /compute/v2.1/os-volumes => generated 345 bytes in 4158 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Sep 08 13:53:31.687014 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d1baa569-6e08-4ee3-846e-5d7dcd47c058 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Calling method '.version_select at 0x7fce02fa0280>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:31.911703 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d1baa569-6e08-4ee3-846e-5d7dcd47c058 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/45f0b9e0-71a5-47f6-af79-b0a25a8105f5" status: 200 len: 345 microversion: 2.1 time: 0.229027 Sep 08 13:53:31.912480 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 2/2] 10.222.0.25 () {58 vars in 1245 bytes} [Thu Sep 8 13:53:31 2022] GET /compute/v2.1/os-volumes/45f0b9e0-71a5-47f6-af79-b0a25a8105f5 => generated 345 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:53:32.189763 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b1bf7fe1-500c-44e1-978c-c162a3f7a032 tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 4.209152 Sep 08 13:53:32.190445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1/3] 10.222.0.25 () {60 vars in 1156 bytes} [Thu Sep 8 13:53:27 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 4211 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) Sep 08 13:53:32.205542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1d14052c-5ecd-4ec2-b409-f8995177501f tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] Calling method '.version_select at 0x7fce02fa0940>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:32.267548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1d14052c-5ecd-4ec2-b409-f8995177501f tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35" status: 200 len: 353 microversion: 2.1 time: 0.063235 Sep 08 13:53:32.267548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 3/4] 10.222.0.25 () {58 vars in 1245 bytes} [Thu Sep 8 13:53:32 2022] GET /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35 => generated 353 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:32.927323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a2c8d2f8-c695-424a-8f27-511367995d22 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Calling method '.version_select at 0x7fce02f9c280>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:32.994722 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a2c8d2f8-c695-424a-8f27-511367995d22 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/45f0b9e0-71a5-47f6-af79-b0a25a8105f5" status: 200 len: 346 microversion: 2.1 time: 0.071407 Sep 08 13:53:33.008997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b1aba8b4-4299-42c4-b835-7089c144c270 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Action: 'create', calling method: .version_select at 0x7fce02f2b040>, body: {"snapshot": {"volume_id": "45f0b9e0-71a5-47f6-af79-b0a25a8105f5", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-517374401"}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:53:33.009854 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 2/5] 10.222.0.25 () {58 vars in 1245 bytes} [Thu Sep 8 13:53:32 2022] GET /compute/v2.1/os-volumes/45f0b9e0-71a5-47f6-af79-b0a25a8105f5 => generated 346 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:53:33.291352 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-962b6954-7eae-4e41-9909-691fd91bde2a tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] Calling method '.version_select at 0x7fce02f9c9d0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:33.377158 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-962b6954-7eae-4e41-9909-691fd91bde2a tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35" status: 200 len: 354 microversion: 2.1 time: 0.099553 Sep 08 13:53:33.380048 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 3/6] 10.222.0.25 () {58 vars in 1245 bytes} [Thu Sep 8 13:53:33 2022] GET /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35 => generated 354 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:33.395606 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0d8dce4d-2676-4b29-80e2-91575b8cbfd5 tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] Calling method '.version_select at 0x7fce02f280d0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:33.483214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0d8dce4d-2676-4b29-80e2-91575b8cbfd5 tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35" status: 200 len: 354 microversion: 2.1 time: 0.095208 Sep 08 13:53:33.484071 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1245 bytes} [Thu Sep 8 13:53:33 2022] GET /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35 => generated 354 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:53:33.502660 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a70fa1f9-5080-4d59-8b0f-76cb1763bae5 tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] Calling method '.version_select at 0x7fce02f9cd30>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:33.654633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b1aba8b4-4299-42c4-b835-7089c144c270 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "POST /compute/v2.1/os-snapshots" status: 200 len: 286 microversion: 2.1 time: 0.649306 Sep 08 13:53:33.662676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 4/8] 10.222.0.25 () {60 vars in 1162 bytes} [Thu Sep 8 13:53:33 2022] POST /compute/v2.1/os-snapshots => generated 286 bytes in 658 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:33.667876 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9df7735b-9180-449a-944f-6e4bf26278c0 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Calling method '.version_select at 0x7fce02f2b8b0>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:33.724506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9df7735b-9180-449a-944f-6e4bf26278c0 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d" status: 200 len: 286 microversion: 2.1 time: 0.060444 Sep 08 13:53:33.731245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 5/9] 10.222.0.25 () {58 vars in 1251 bytes} [Thu Sep 8 13:53:33 2022] GET /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d => generated 286 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:33.736652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6a22d866-c226-4a63-8ff3-02bd7aaf452e tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Calling method '.version_select at 0x7fce02f2b1f0>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:33.838212 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6a22d866-c226-4a63-8ff3-02bd7aaf452e tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots" status: 200 len: 289 microversion: 2.1 time: 0.105631 Sep 08 13:53:33.838788 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 6/10] 10.222.0.25 () {58 vars in 1140 bytes} [Thu Sep 8 13:53:33 2022] GET /compute/v2.1/os-snapshots => generated 289 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:33.852911 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4d153a66-b272-4df1-b471-0d479451a7df tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Calling method '.version_select at 0x7fce02f9ed30>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:33.935109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4d153a66-b272-4df1-b471-0d479451a7df tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d" status: 200 len: 286 microversion: 2.1 time: 0.087658 Sep 08 13:53:33.935903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 7/11] 10.222.0.25 () {58 vars in 1251 bytes} [Thu Sep 8 13:53:33 2022] GET /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d => generated 286 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:34.018295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a70fa1f9-5080-4d59-8b0f-76cb1763bae5 tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35" status: 202 len: 0 microversion: 2.1 time: 0.517665 Sep 08 13:53:34.020083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 5/12] 10.222.0.25 () {58 vars in 1248 bytes} [Thu Sep 8 13:53:33 2022] DELETE /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35 => generated 0 bytes in 523 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 13:53:34.030156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c6a49db9-ad1d-4bc4-a0c6-a1a95de66f32 tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] Calling method '.version_select at 0x7fce02f9e700>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:34.052997 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d80fb8df-ab61-4cfb-a77c-666980365899 tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] Calling method '.version_select at 0x7fce02f9c5e0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:34.192212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d80fb8df-ab61-4cfb-a77c-666980365899 tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/93c4ba24-899c-461b-86db-18396ff1c0d0" status: 200 len: 351 microversion: 2.1 time: 0.143337 Sep 08 13:53:34.193439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 6/13] 10.222.0.25 () {58 vars in 1245 bytes} [Thu Sep 8 13:53:34 2022] GET /compute/v2.1/os-volumes/93c4ba24-899c-461b-86db-18396ff1c0d0 => generated 351 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:34.197987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c6a49db9-ad1d-4bc4-a0c6-a1a95de66f32 tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35" status: 200 len: 353 microversion: 2.1 time: 0.173060 Sep 08 13:53:34.198712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 8/14] 10.222.0.25 () {58 vars in 1245 bytes} [Thu Sep 8 13:53:34 2022] GET /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35 => generated 353 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:34.950549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8fdc49b6-2b4d-4308-a085-13078dd86661 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Calling method '.version_select at 0x7fce02f281f0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:35.004577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8fdc49b6-2b4d-4308-a085-13078dd86661 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d" status: 200 len: 286 microversion: 2.1 time: 0.060151 Sep 08 13:53:35.005160 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 7/15] 10.222.0.25 () {58 vars in 1251 bytes} [Thu Sep 8 13:53:34 2022] GET /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d => generated 286 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:35.212848 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4eeb06a8-47fe-49fe-800d-5824986cfa5a tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] Calling method '.version_select at 0x7fce02f9e040>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:35.348986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4eeb06a8-47fe-49fe-800d-5824986cfa5a tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35" status: 200 len: 353 microversion: 2.1 time: 0.139816 Sep 08 13:53:35.349555 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 9/16] 10.222.0.25 () {58 vars in 1245 bytes} [Thu Sep 8 13:53:35 2022] GET /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35 => generated 353 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:36.022165 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0b2a18db-f960-4c64-85da-38661a0d7faf tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Calling method '.version_select at 0x7fce02f9c430>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:36.212357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0b2a18db-f960-4c64-85da-38661a0d7faf tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d" status: 200 len: 286 microversion: 2.1 time: 0.194709 Sep 08 13:53:36.218568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 8/17] 10.222.0.25 () {58 vars in 1251 bytes} [Thu Sep 8 13:53:36 2022] GET /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d => generated 286 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:36.366623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8c49a2df-ad36-41db-918f-655aad857891 tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] Calling method '.version_select at 0x7fce02f2bd30>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:36.517167 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8c49a2df-ad36-41db-918f-655aad857891 tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35" status: 200 len: 353 microversion: 2.1 time: 0.157133 Sep 08 13:53:36.517725 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 10/18] 10.222.0.25 () {58 vars in 1245 bytes} [Thu Sep 8 13:53:36 2022] GET /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35 => generated 353 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:37.227289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d1556bbd-f116-43fd-a257-fe859e1c6c37 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Calling method '.version_select at 0x7fce02f9cb80>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:37.354896 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d1556bbd-f116-43fd-a257-fe859e1c6c37 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d" status: 200 len: 286 microversion: 2.1 time: 0.130319 Sep 08 13:53:37.363170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 9/19] 10.222.0.25 () {58 vars in 1251 bytes} [Thu Sep 8 13:53:37 2022] GET /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d => generated 286 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:37.531586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3b5fc412-f386-4711-b824-ce6214fe7293 tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] Calling method '.version_select at 0x7fce02f9eaf0>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:37.650794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3b5fc412-f386-4711-b824-ce6214fe7293 tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35" status: 200 len: 353 microversion: 2.1 time: 0.123539 Sep 08 13:53:37.652297 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 11/20] 10.222.0.25 () {58 vars in 1245 bytes} [Thu Sep 8 13:53:37 2022] GET /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35 => generated 353 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:38.368379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-166604fb-64f5-420d-9787-b3bd1f1778b0 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Calling method '.version_select at 0x7fce02f28550>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:38.433868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-166604fb-64f5-420d-9787-b3bd1f1778b0 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d" status: 200 len: 287 microversion: 2.1 time: 0.069858 Sep 08 13:53:38.444358 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 10/21] 10.222.0.25 () {58 vars in 1251 bytes} [Thu Sep 8 13:53:38 2022] GET /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d => generated 287 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:38.448969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ca5bf0ac-91d3-4ce4-8b97-7a3043b31e38 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Calling method '.version_select at 0x7fce02f9e790>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:38.466164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8191019b-afcb-4a5d-b1df-389c0e42eb57 tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] Calling method '.version_select at 0x7fce02f288b0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:38.531339 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8191019b-afcb-4a5d-b1df-389c0e42eb57 tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/e32a66da-7859-4545-a3f0-e402d07dddcb" status: 200 len: 351 microversion: 2.1 time: 0.070239 Sep 08 13:53:38.532687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 11/22] 10.222.0.25 () {58 vars in 1245 bytes} [Thu Sep 8 13:53:38 2022] GET /compute/v2.1/os-volumes/e32a66da-7859-4545-a3f0-e402d07dddcb => generated 351 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:38.665556 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-34eedb11-3fe1-49cd-b3fe-c67756147efb tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] Calling method '.version_select at 0x7fce02f9cca0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:38.721669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ca5bf0ac-91d3-4ce4-8b97-7a3043b31e38 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "DELETE /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d" status: 202 len: 0 microversion: 2.1 time: 0.276828 Sep 08 13:53:38.722311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 12/23] 10.222.0.25 () {58 vars in 1254 bytes} [Thu Sep 8 13:53:38 2022] DELETE /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d => generated 0 bytes in 278 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 08 13:53:38.730462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0625a751-efe7-41dd-8a4d-dd7767226dd8 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Calling method '.version_select at 0x7fce02f2b280>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:38.794270 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-34eedb11-3fe1-49cd-b3fe-c67756147efb tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] HTTP exception thrown: Volume 2938ea1a-ca19-4b76-83e3-534d79e7de35 could not be found. Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-34eedb11-3fe1-49cd-b3fe-c67756147efb tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] Request method failure captured: Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35 HTTP/1.1 Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPXjxvbcC-VeRWJ6Tvx55LOUyFvi-ZjMM89ou4m6h5aKyJzMjwQ3F52tItJQRPeFIJW_37wVMPdazIxC6QQvQ5JT72iYS6bMnpJN-LuerkJD4jgjwBs3TYDtyUd5PQmn9dfkvs66ITuMdA2Ewq8dKjKZigwPjU7zYT4-i8BH_2B2dk Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 7538afd39b364087a38b64080f27ec58 Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-VolumesGetTestJSON-1429744866 Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 13:53:38.798793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 13:53:38.799989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_7538afd39b364087a38b64080f27ec58"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/7538afd39b364087a38b64080f27ec58"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/7538afd39b364087a38b64080f27ec58"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/7538afd39b364087a38b64080f27ec58"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:53:38.799989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-VolumesGetTestJSON-1429744866 Sep 08 13:53:38.799989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 7538afd39b364087a38b64080f27ec58 Sep 08 13:53:38.799989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-VolumesGetTestJSON-1429744866 Sep 08 13:53:38.799989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-VolumesGetTestJSON-1429744866-project Sep 08 13:53:38.799989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 13:53:38.799989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 13:53:38.799989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: bfc74593ee634c619b783f9daa20312c Sep 08 13:53:38.799989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-VolumesGetTestJSON-1429744866-project Sep 08 13:53:38.799989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 13:53:38.799989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Volume 2938ea1a-ca19-4b76-83e3-534d79e7de35 could not be found. Sep 08 13:53:38.799989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '2938ea1a-ca19-4b76-83e3-534d79e7de35'} Sep 08 13:53:38.799989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 2938ea1a-ca19-4b76-83e3-534d79e7de35 could not be found. (HTTP 404) (Request-ID: req-9438870e-6f97-4843-97e3-a8582b4298b8) Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:53:38.800700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 08 13:53:38.801667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 2938ea1a-ca19-4b76-83e3-534d79e7de35 could not be found. Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 08 13:53:38.802732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 13:53:38.803691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 2938ea1a-ca19-4b76-83e3-534d79e7de35 could not be found. Sep 08 13:53:38.803691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:38.803691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:53:38.803691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:38.803691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:38.803691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:53:38.803691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:53:38.803691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:53:38.803691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:53:38.803691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 2938ea1a-ca19-4b76-83e3-534d79e7de35 could not be found. Sep 08 13:53:38.803691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:38.803691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-34eedb11-3fe1-49cd-b3fe-c67756147efb tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] Returning 404 to user: Volume 2938ea1a-ca19-4b76-83e3-534d79e7de35 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:53:38.803691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-34eedb11-3fe1-49cd-b3fe-c67756147efb tempest-VolumesGetTestJSON-1429744866 tempest-VolumesGetTestJSON-1429744866-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35" status: 404 len: 109 microversion: 2.1 time: 0.139712 Sep 08 13:53:38.803691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 12/24] 10.222.0.25 () {58 vars in 1245 bytes} [Thu Sep 8 13:53:38 2022] GET /compute/v2.1/os-volumes/2938ea1a-ca19-4b76-83e3-534d79e7de35 => generated 109 bytes in 142 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 08 13:53:38.883215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0625a751-efe7-41dd-8a4d-dd7767226dd8 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d" status: 200 len: 286 microversion: 2.1 time: 0.156526 Sep 08 13:53:38.884652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 13/25] 10.222.0.25 () {58 vars in 1251 bytes} [Thu Sep 8 13:53:38 2022] GET /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d => generated 286 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:39.901396 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a046b233-335b-48d6-a63c-faccf713dd68 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Calling method '.version_select at 0x7fce038ec9d0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:39.951596 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-a046b233-335b-48d6-a63c-faccf713dd68 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] HTTP exception thrown: Snapshot a9ac24ee-172c-4b10-adec-1ac4d653e08d could not be found. Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a046b233-335b-48d6-a63c-faccf713dd68 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Request method failure captured: Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d HTTP/1.1 Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPXLRzdyugKanJnS2jTss2KjTbEmabgAUhDEJqOzZXZnVwfD9DUDj4pqCoTBEIGZJ-sCg1_7sLLYL4MJJtbFV3hqrQovV_RVhn5XnYU2j4lf0WQcXEBTx34JaWo-6KWnGAuXNFxsxAWqx44JFEQSt5CPO_stlawwtOU7KooKj_1ZjE Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 7711a822e7ef4fbda3d19219739b228e Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1293601293 Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 13:53:39.954129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 13:53:39.955434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_7711a822e7ef4fbda3d19219739b228e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/7711a822e7ef4fbda3d19219739b228e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/7711a822e7ef4fbda3d19219739b228e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/7711a822e7ef4fbda3d19219739b228e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:53:39.955434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1293601293 Sep 08 13:53:39.955434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 7711a822e7ef4fbda3d19219739b228e Sep 08 13:53:39.955434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1293601293 Sep 08 13:53:39.955434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-VolumesSnapshotsTestJSON-1293601293-project Sep 08 13:53:39.955434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 13:53:39.955434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 13:53:39.955434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 9b91f50cffd149f29d30d68083f749dd Sep 08 13:53:39.955434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1293601293-project Sep 08 13:53:39.955434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 13:53:39.955434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Snapshot a9ac24ee-172c-4b10-adec-1ac4d653e08d could not be found. Sep 08 13:53:39.955434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'a9ac24ee-172c-4b10-adec-1ac4d653e08d'} Sep 08 13:53:39.955434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:53:39.956253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:39.956253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 08 13:53:39.956253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 08 13:53:39.956253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 08 13:53:39.956253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 08 13:53:39.956253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 08 13:53:39.956253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 08 13:53:39.956253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 08 13:53:39.956253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 08 13:53:39.956253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 08 13:53:39.956253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 08 13:53:39.956253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 13:53:39.956253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 13:53:39.956253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 13:53:39.956253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 13:53:39.956253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot a9ac24ee-172c-4b10-adec-1ac4d653e08d could not be found. (HTTP 404) (Request-ID: req-ac77099d-5e3a-4597-bd7b-dacd60064e9d) Sep 08 13:53:39.956253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:39.956253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 08 13:53:39.957304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot a9ac24ee-172c-4b10-adec-1ac4d653e08d could not be found. Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Sep 08 13:53:39.959072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 13:53:39.961522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot a9ac24ee-172c-4b10-adec-1ac4d653e08d could not be found. Sep 08 13:53:39.961522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:39.961522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:53:39.961522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:39.961522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:39.961522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:53:39.961522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:53:39.961522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:53:39.961522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:53:39.961522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot a9ac24ee-172c-4b10-adec-1ac4d653e08d could not be found. Sep 08 13:53:39.961522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:39.966345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a046b233-335b-48d6-a63c-faccf713dd68 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Returning 404 to user: Snapshot a9ac24ee-172c-4b10-adec-1ac4d653e08d could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:53:39.967459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a046b233-335b-48d6-a63c-faccf713dd68 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "GET /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d" status: 404 len: 111 microversion: 2.1 time: 0.074900 Sep 08 13:53:39.972027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 13/26] 10.222.0.25 () {58 vars in 1251 bytes} [Thu Sep 8 13:53:39 2022] GET /compute/v2.1/os-snapshots/a9ac24ee-172c-4b10-adec-1ac4d653e08d => generated 111 bytes in 80 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 13:53:39.986390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6c50835d-3518-4838-90cd-853699541ec8 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Calling method '.version_select at 0x7fce02f2b670>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:40.255707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6c50835d-3518-4838-90cd-853699541ec8 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/45f0b9e0-71a5-47f6-af79-b0a25a8105f5" status: 202 len: 0 microversion: 2.1 time: 0.282329 Sep 08 13:53:40.260187 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 14/27] 10.222.0.25 () {58 vars in 1248 bytes} [Thu Sep 8 13:53:39 2022] DELETE /compute/v2.1/os-volumes/45f0b9e0-71a5-47f6-af79-b0a25a8105f5 => generated 0 bytes in 284 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 13:53:40.269211 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-622df879-34f6-4244-83a3-4dbafecc6348 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Calling method '.version_select at 0x7fce02f174c0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:40.331826 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-622df879-34f6-4244-83a3-4dbafecc6348 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/45f0b9e0-71a5-47f6-af79-b0a25a8105f5" status: 200 len: 345 microversion: 2.1 time: 0.068184 Sep 08 13:53:40.341804 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 14/28] 10.222.0.25 () {58 vars in 1245 bytes} [Thu Sep 8 13:53:40 2022] GET /compute/v2.1/os-volumes/45f0b9e0-71a5-47f6-af79-b0a25a8105f5 => generated 345 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:40.577326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-eccb692b-0a09-4110-8297-14023a2c5617 tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] Calling method '.version_select at 0x7fce02f9e700>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:40.657060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-eccb692b-0a09-4110-8297-14023a2c5617 tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/0c4bdc44-1f7d-47f4-bb70-3a0c5ea3d7cf" status: 200 len: 350 microversion: 2.1 time: 0.086803 Sep 08 13:53:40.658316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 15/29] 10.222.0.25 () {58 vars in 1245 bytes} [Thu Sep 8 13:53:40 2022] GET /compute/v2.1/os-volumes/0c4bdc44-1f7d-47f4-bb70-3a0c5ea3d7cf => generated 350 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:40.671588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-272e1018-5a41-4c75-99ef-756a33f0a47a tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] Calling method '.version_select at 0x7fce02f17af0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:40.897525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-272e1018-5a41-4c75-99ef-756a33f0a47a tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] 10.222.0.25 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.229863 Sep 08 13:53:40.899229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 15/30] 10.222.0.25 () {58 vars in 1134 bytes} [Thu Sep 8 13:53:40 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:40.916554 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1cb760f3-cfb8-4ef0-90ad-f65fe955471f tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] Calling method '.version_select at 0x7fce02f9e160>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:41.072724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1cb760f3-cfb8-4ef0-90ad-f65fe955471f tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] 10.222.0.25 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.160967 Sep 08 13:53:41.087675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 16/31] 10.222.0.25 () {58 vars in 1149 bytes} [Thu Sep 8 13:53:40 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 694 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:41.092113 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-99be403f-7b81-4fff-899e-69874db42d84 tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] Calling method '.version_select at 0x7fce030be430>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:41.295070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-99be403f-7b81-4fff-899e-69874db42d84 tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] 10.222.0.25 "GET /compute/v2.1/os-volumes" status: 200 len: 1035 microversion: 2.1 time: 0.207257 Sep 08 13:53:41.296141 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 16/32] 10.222.0.25 () {58 vars in 1134 bytes} [Thu Sep 8 13:53:41 2022] GET /compute/v2.1/os-volumes => generated 1035 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:53:41.311049 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-300fcb52-baf6-41ec-8a44-308f744cff8e tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] Calling method '.version_select at 0x7fce02f2b040>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:41.347441 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-51c65a66-f3cb-43f7-8448-0d2db0b0b01b tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Calling method '.version_select at 0x7fce02f1c8b0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:41.384562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-51c65a66-f3cb-43f7-8448-0d2db0b0b01b tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] HTTP exception thrown: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-51c65a66-f3cb-43f7-8448-0d2db0b0b01b tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Request method failure captured: Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/os-volumes/45f0b9e0-71a5-47f6-af79-b0a25a8105f5 HTTP/1.1 Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPXLRzdyugKanJnS2jTss2KjTbEmabgAUhDEJqOzZXZnVwfD9DUDj4pqCoTBEIGZJ-sCg1_7sLLYL4MJJtbFV3hqrQovV_RVhn5XnYU2j4lf0WQcXEBTx34JaWo-6KWnGAuXNFxsxAWqx44JFEQSt5CPO_stlawwtOU7KooKj_1ZjE Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 7711a822e7ef4fbda3d19219739b228e Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1293601293 Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 13:53:41.387903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 13:53:41.399965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_7711a822e7ef4fbda3d19219739b228e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/7711a822e7ef4fbda3d19219739b228e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/7711a822e7ef4fbda3d19219739b228e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/7711a822e7ef4fbda3d19219739b228e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:53:41.399965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1293601293 Sep 08 13:53:41.399965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 7711a822e7ef4fbda3d19219739b228e Sep 08 13:53:41.399965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1293601293 Sep 08 13:53:41.399965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-VolumesSnapshotsTestJSON-1293601293-project Sep 08 13:53:41.399965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 13:53:41.399965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 13:53:41.399965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 9b91f50cffd149f29d30d68083f749dd Sep 08 13:53:41.399965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1293601293-project Sep 08 13:53:41.399965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 13:53:41.399965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. Sep 08 13:53:41.399965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '45f0b9e0-71a5-47f6-af79-b0a25a8105f5'} Sep 08 13:53:41.399965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. (HTTP 404) (Request-ID: req-68e3d9f7-af95-4e2c-b36f-75733d28b3fa) Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:53:41.400811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 08 13:53:41.402177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 08 13:53:41.403946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 13:53:41.405074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. Sep 08 13:53:41.405074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.405074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:53:41.405074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.405074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:41.405074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:53:41.405074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:53:41.405074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:53:41.405074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:53:41.405074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. Sep 08 13:53:41.405074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.405074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-51c65a66-f3cb-43f7-8448-0d2db0b0b01b tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Returning 404 to user: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:53:41.405074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-51c65a66-f3cb-43f7-8448-0d2db0b0b01b tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/45f0b9e0-71a5-47f6-af79-b0a25a8105f5" status: 404 len: 109 microversion: 2.1 time: 0.046431 Sep 08 13:53:41.405074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 17/33] 10.222.0.25 () {58 vars in 1245 bytes} [Thu Sep 8 13:53:41 2022] GET /compute/v2.1/os-volumes/45f0b9e0-71a5-47f6-af79-b0a25a8105f5 => generated 109 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 08 13:53:41.412937 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2b24486f-35b9-483c-93cf-de8f833ab966 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Calling method '.version_select at 0x7fce02f1c9d0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:41.462892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-2b24486f-35b9-483c-93cf-de8f833ab966 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] HTTP exception thrown: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2b24486f-35b9-483c-93cf-de8f833ab966 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Request method failure captured: Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/os-volumes/45f0b9e0-71a5-47f6-af79-b0a25a8105f5 HTTP/1.1 Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPXLRzdyugKanJnS2jTss2KjTbEmabgAUhDEJqOzZXZnVwfD9DUDj4pqCoTBEIGZJ-sCg1_7sLLYL4MJJtbFV3hqrQovV_RVhn5XnYU2j4lf0WQcXEBTx34JaWo-6KWnGAuXNFxsxAWqx44JFEQSt5CPO_stlawwtOU7KooKj_1ZjE Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 7711a822e7ef4fbda3d19219739b228e Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1293601293 Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 13:53:41.464930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 13:53:41.466712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_7711a822e7ef4fbda3d19219739b228e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/7711a822e7ef4fbda3d19219739b228e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/7711a822e7ef4fbda3d19219739b228e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/7711a822e7ef4fbda3d19219739b228e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:53:41.466712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1293601293 Sep 08 13:53:41.466712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 7711a822e7ef4fbda3d19219739b228e Sep 08 13:53:41.466712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1293601293 Sep 08 13:53:41.466712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-VolumesSnapshotsTestJSON-1293601293-project Sep 08 13:53:41.466712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 13:53:41.466712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 13:53:41.466712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 9b91f50cffd149f29d30d68083f749dd Sep 08 13:53:41.466712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1293601293-project Sep 08 13:53:41.466712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 13:53:41.466712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. Sep 08 13:53:41.466712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '45f0b9e0-71a5-47f6-af79-b0a25a8105f5'} Sep 08 13:53:41.466712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. (HTTP 404) (Request-ID: req-e3ca7353-0b29-4726-a3c0-44c57bd7711e) Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:53:41.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 08 13:53:41.469368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 08 13:53:41.471957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 13:53:41.474292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. Sep 08 13:53:41.474292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.474292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:53:41.474292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.474292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:41.474292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:53:41.474292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:53:41.474292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:53:41.474292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:53:41.474292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. Sep 08 13:53:41.474292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.474292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2b24486f-35b9-483c-93cf-de8f833ab966 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Returning 404 to user: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:53:41.474292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2b24486f-35b9-483c-93cf-de8f833ab966 tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/45f0b9e0-71a5-47f6-af79-b0a25a8105f5" status: 404 len: 109 microversion: 2.1 time: 0.059949 Sep 08 13:53:41.474292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 18/34] 10.222.0.25 () {58 vars in 1248 bytes} [Thu Sep 8 13:53:41 2022] DELETE /compute/v2.1/os-volumes/45f0b9e0-71a5-47f6-af79-b0a25a8105f5 => generated 109 bytes in 72 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 13:53:41.477672 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-eb593e2f-0397-4580-bbf9-35fc3759967c tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Calling method '.version_select at 0x7fce02f9cca0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:41.544608 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-eb593e2f-0397-4580-bbf9-35fc3759967c tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] HTTP exception thrown: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-eb593e2f-0397-4580-bbf9-35fc3759967c tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Request method failure captured: Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/os-volumes/45f0b9e0-71a5-47f6-af79-b0a25a8105f5 HTTP/1.1 Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPXLRzdyugKanJnS2jTss2KjTbEmabgAUhDEJqOzZXZnVwfD9DUDj4pqCoTBEIGZJ-sCg1_7sLLYL4MJJtbFV3hqrQovV_RVhn5XnYU2j4lf0WQcXEBTx34JaWo-6KWnGAuXNFxsxAWqx44JFEQSt5CPO_stlawwtOU7KooKj_1ZjE Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 7711a822e7ef4fbda3d19219739b228e Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1293601293 Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 13:53:41.547308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 13:53:41.553580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_7711a822e7ef4fbda3d19219739b228e"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/7711a822e7ef4fbda3d19219739b228e"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/7711a822e7ef4fbda3d19219739b228e"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/7711a822e7ef4fbda3d19219739b228e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:53:41.553580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1293601293 Sep 08 13:53:41.553580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 7711a822e7ef4fbda3d19219739b228e Sep 08 13:53:41.553580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1293601293 Sep 08 13:53:41.553580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-VolumesSnapshotsTestJSON-1293601293-project Sep 08 13:53:41.553580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 13:53:41.553580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 13:53:41.553580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 9b91f50cffd149f29d30d68083f749dd Sep 08 13:53:41.553580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1293601293-project Sep 08 13:53:41.553580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 13:53:41.553580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. Sep 08 13:53:41.553580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '45f0b9e0-71a5-47f6-af79-b0a25a8105f5'} Sep 08 13:53:41.553580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. (HTTP 404) (Request-ID: req-d2eb028b-5732-4111-a9f0-b4a0cb1e9cd9) Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:53:41.555363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 08 13:53:41.556569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 08 13:53:41.557650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 13:53:41.559321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. Sep 08 13:53:41.559321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.559321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:53:41.559321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.559321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:53:41.559321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:53:41.559321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:53:41.559321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:53:41.559321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:53:41.559321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. Sep 08 13:53:41.559321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:53:41.559321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-eb593e2f-0397-4580-bbf9-35fc3759967c tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] Returning 404 to user: Volume 45f0b9e0-71a5-47f6-af79-b0a25a8105f5 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:53:41.559321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-eb593e2f-0397-4580-bbf9-35fc3759967c tempest-VolumesSnapshotsTestJSON-1293601293 tempest-VolumesSnapshotsTestJSON-1293601293-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/45f0b9e0-71a5-47f6-af79-b0a25a8105f5" status: 404 len: 109 microversion: 2.1 time: 0.075693 Sep 08 13:53:41.563908 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 19/35] 10.222.0.25 () {58 vars in 1245 bytes} [Thu Sep 8 13:53:41 2022] GET /compute/v2.1/os-volumes/45f0b9e0-71a5-47f6-af79-b0a25a8105f5 => generated 109 bytes in 91 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 13:53:41.599232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-300fcb52-baf6-41ec-8a44-308f744cff8e tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] 10.222.0.25 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.293541 Sep 08 13:53:41.600067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 17/36] 10.222.0.25 () {58 vars in 1167 bytes} [Thu Sep 8 13:53:41 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 354 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:41.615867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d80d1a8e-fcdd-4264-beb8-7b10e33c2ac7 tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] Calling method '.version_select at 0x7fce02f1c5e0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:41.765018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d80d1a8e-fcdd-4264-beb8-7b10e33c2ac7 tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 694 microversion: 2.1 time: 0.152922 Sep 08 13:53:41.765595 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 20/37] 10.222.0.25 () {58 vars in 1170 bytes} [Thu Sep 8 13:53:41 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 694 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:41.784945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bc8c4dac-7ba9-4304-9100-196cad2d74a4 tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] Calling method '.version_select at 0x7fce02f2bca0>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:42.013257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bc8c4dac-7ba9-4304-9100-196cad2d74a4 tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.236114 Sep 08 13:53:42.013826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 18/38] 10.222.0.25 () {58 vars in 1155 bytes} [Thu Sep 8 13:53:41 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:42.027939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b65aceef-8ad8-4f6c-af8a-3ee85e23e579 tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] Calling method '.version_select at 0x7fce02f9c1f0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:42.262887 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b65aceef-8ad8-4f6c-af8a-3ee85e23e579 tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 354 microversion: 2.1 time: 0.239094 Sep 08 13:53:42.263772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 21/39] 10.222.0.25 () {58 vars in 1188 bytes} [Thu Sep 8 13:53:42 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 354 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:42.278878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-656261d5-f56b-42cb-a627-d8867f078e6e tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] Calling method '.version_select at 0x7fce02f9e430>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:42.507170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-656261d5-f56b-42cb-a627-d8867f078e6e tempest-VolumesTestJSON-201519389 tempest-VolumesTestJSON-201519389-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1035 microversion: 2.1 time: 0.233326 Sep 08 13:53:42.508200 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 19/40] 10.222.0.25 () {58 vars in 1155 bytes} [Thu Sep 8 13:53:42 2022] GET /compute/v2.1/os-volumes/detail => generated 1035 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:44.261800 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7acddf8b-babd-4cc6-8133-66d97f3ef6f3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Action: 'create', calling method: .version_select at 0x7fce02f238b0>, body: {"keypair": {"name": "tempest-keypair-494104329", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLxpKlGHkWpxQOTbH7TMW9EsC1oMM9zogxT/b6APPyZZp55+HS1HyHN18QLRPwu92BorFPlyOMmOJ3oHUAkrPU5HNmUJX21OW1C4JJ4jlT4CnbEpuIYpNJwQ/RgdttHLTQ=="}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:53:44.386672 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-7acddf8b-babd-4cc6-8133-66d97f3ef6f3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Getting quotas for project a4e0fba6cab34c429c73fcfac7ebfd94. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:53:44.397584 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-7acddf8b-babd-4cc6-8133-66d97f3ef6f3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Getting quotas for user eca89f435299415c8448931e73f5b0fc and project a4e0fba6cab34c429c73fcfac7ebfd94. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:53:44.648683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7acddf8b-babd-4cc6-8133-66d97f3ef6f3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 1.128618 Sep 08 13:53:44.656348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 22/41] 10.222.0.25 () {60 vars in 1159 bytes} [Thu Sep 8 13:53:43 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 1131 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:53:45.326865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b026537f-7010-4a2c-9935-6f2f9899aa29 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1235447245", "imageRef": "20b30425-92e5-4d78-8502-45557526ea49", "flavorRef": "452", "networks": [{"uuid": "c739a965-08b7-4404-8bde-85db5e6ad487"}]}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:53:45.567945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.network.neutron [None req-b026537f-7010-4a2c-9935-6f2f9899aa29 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] validate_networks() for [('c739a965-08b7-4404-8bde-85db5e6ad487', None, None, None, None, None)] {{(pid=127601) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 13:53:46.280514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4911f2a5-994b-4188-8433-194898dbb904 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Action: 'create', calling method: .version_select at 0x7fce02e780d0>, body: {"keypair": {"name": "tempest-keypair-920229013", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBFCCRhVasPb69s0z5sbLTr510F0d3EISALQb3dekVKUZX2krLfY+tkGoflS6Ir+ToQbKofoQmRHG2GgFVHym1kESBtjgSjRZqpKOtR8/d43mJUzTML7s1r2N+BAkn80SLA=="}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:53:46.321693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-4911f2a5-994b-4188-8433-194898dbb904 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Getting quotas for project 4c194dd986d442d3bcaa83442a49ff70. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:53:46.341115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-4911f2a5-994b-4188-8433-194898dbb904 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Getting quotas for user e9925313d1d446fa81a00cb55348dff7 and project 4c194dd986d442d3bcaa83442a49ff70. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:53:46.535978 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4911f2a5-994b-4188-8433-194898dbb904 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.905600 Sep 08 13:53:46.536779 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 23/42] 10.222.0.25 () {62 vars in 1200 bytes} [Thu Sep 8 13:53:45 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 907 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 08 13:53:47.044189 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3b786073-1deb-4d72-b6e7-a67e6af0a025 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1724175055", "imageRef": "20b30425-92e5-4d78-8502-45557526ea49", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:53:47.290545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.network.neutron [None req-3b786073-1deb-4d72-b6e7-a67e6af0a025 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] validate_networks() for [('000958f9-5cdd-4954-9b31-e6534b51d985', None, None, None, None, None)] {{(pid=127602) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 13:53:48.329146 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-b026537f-7010-4a2c-9935-6f2f9899aa29 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Flavor limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 13:53:48.329720 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-b026537f-7010-4a2c-9935-6f2f9899aa29 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Image limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 13:53:48.330022 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-b026537f-7010-4a2c-9935-6f2f9899aa29 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Flavor pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 13:53:48.330801 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-b026537f-7010-4a2c-9935-6f2f9899aa29 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Image pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 13:53:48.331365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-b026537f-7010-4a2c-9935-6f2f9899aa29 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 13:53:48.367932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b026537f-7010-4a2c-9935-6f2f9899aa29 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:48.368597 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b026537f-7010-4a2c-9935-6f2f9899aa29 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:48.369190 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b026537f-7010-4a2c-9935-6f2f9899aa29 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:48.373445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b026537f-7010-4a2c-9935-6f2f9899aa29 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:48.374374 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b026537f-7010-4a2c-9935-6f2f9899aa29 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:48.374976 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b026537f-7010-4a2c-9935-6f2f9899aa29 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:48.447946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-b026537f-7010-4a2c-9935-6f2f9899aa29 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Getting quotas for project 5775050045e749b1bdb26dddcce9a8ff. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:53:48.460970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-b026537f-7010-4a2c-9935-6f2f9899aa29 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Getting quotas for user 8294dcc1178045abbcf7a497564eb122 and project 5775050045e749b1bdb26dddcce9a8ff. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:53:48.536091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b026537f-7010-4a2c-9935-6f2f9899aa29 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Going to run 1 instances... {{(pid=127601) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 13:53:48.539648 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b026537f-7010-4a2c-9935-6f2f9899aa29 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='20b30425-92e5-4d78-8502-45557526ea49',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127601) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 13:53:48.862348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b026537f-7010-4a2c-9935-6f2f9899aa29 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 3.846190 Sep 08 13:53:48.865772 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 20/43] 10.222.0.25 () {60 vars in 1147 bytes} [Thu Sep 8 13:53:45 2022] POST /compute/v2.1/servers => generated 380 bytes in 3858 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 08 13:53:48.886620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ab39b812-6fac-449e-8927-74abb7e37a9e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:48.887291 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ab39b812-6fac-449e-8927-74abb7e37a9e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:49.166055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ab39b812-6fac-449e-8927-74abb7e37a9e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:49.196430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ab39b812-6fac-449e-8927-74abb7e37a9e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1215 microversion: 2.1 time: 0.320676 Sep 08 13:53:49.197245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 21/44] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:53:48 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1215 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:50.223416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f249dd80-0629-43f7-8eb1-3035ee4ccf6f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:50.224727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f249dd80-0629-43f7-8eb1-3035ee4ccf6f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:50.280780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-3b786073-1deb-4d72-b6e7-a67e6af0a025 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Flavor limits 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 13:53:50.281303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-3b786073-1deb-4d72-b6e7-a67e6af0a025 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Image limits 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 13:53:50.281842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-3b786073-1deb-4d72-b6e7-a67e6af0a025 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Flavor pref 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 13:53:50.283117 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-3b786073-1deb-4d72-b6e7-a67e6af0a025 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Image pref 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 13:53:50.283625 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-3b786073-1deb-4d72-b6e7-a67e6af0a025 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 13:53:50.318000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b786073-1deb-4d72-b6e7-a67e6af0a025 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:50.335288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b786073-1deb-4d72-b6e7-a67e6af0a025 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:50.337546 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b786073-1deb-4d72-b6e7-a67e6af0a025 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:50.337546 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b786073-1deb-4d72-b6e7-a67e6af0a025 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:50.343108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b786073-1deb-4d72-b6e7-a67e6af0a025 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:50.343715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b786073-1deb-4d72-b6e7-a67e6af0a025 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:50.405548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f249dd80-0629-43f7-8eb1-3035ee4ccf6f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:50.411723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-3b786073-1deb-4d72-b6e7-a67e6af0a025 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Getting quotas for project 1346d4105dce4c6e928fcb11cde56854. Resources: {'cores', 'instances', 'ram'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:53:50.415735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f249dd80-0629-43f7-8eb1-3035ee4ccf6f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1215 microversion: 2.1 time: 0.197481 Sep 08 13:53:50.418940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 22/45] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:53:50 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1215 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:50.453354 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-3b786073-1deb-4d72-b6e7-a67e6af0a025 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Getting quotas for user 792808dcbf894aa98c671cf43ac21b8b and project 1346d4105dce4c6e928fcb11cde56854. Resources: {'cores', 'instances', 'ram'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:53:50.517576 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3b786073-1deb-4d72-b6e7-a67e6af0a025 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Going to run 1 instances... {{(pid=127602) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 13:53:50.527945 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3b786073-1deb-4d72-b6e7-a67e6af0a025 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='20b30425-92e5-4d78-8502-45557526ea49',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127602) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 13:53:50.930724 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3b786073-1deb-4d72-b6e7-a67e6af0a025 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 4.325140 Sep 08 13:53:50.930724 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 24/46] 10.222.0.25 () {60 vars in 1147 bytes} [Thu Sep 8 13:53:46 2022] POST /compute/v2.1/servers => generated 383 bytes in 4328 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 08 13:53:50.933106 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c01277ff-0a8d-4414-94e4-d7eb091945ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:50.934664 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c01277ff-0a8d-4414-94e4-d7eb091945ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:51.022938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c01277ff-0a8d-4414-94e4-d7eb091945ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:51.031745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c01277ff-0a8d-4414-94e4-d7eb091945ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1222 microversion: 2.1 time: 0.102336 Sep 08 13:53:51.032679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 23/47] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:53:50 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1222 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:51.446051 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9e43ba3e-cd9f-4854-b839-499106cfeb55 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:51.446912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9e43ba3e-cd9f-4854-b839-499106cfeb55 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:51.554072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-205a8ad9-9269-41ee-b093-9e05561b4dca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-239020158", "imageRef": "20b30425-92e5-4d78-8502-45557526ea49", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:53:51.795933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.network.neutron [None req-205a8ad9-9269-41ee-b093-9e05561b4dca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] validate_networks() for [('000958f9-5cdd-4954-9b31-e6534b51d985', None, None, None, None, None)] {{(pid=127601) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 13:53:51.894087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9e43ba3e-cd9f-4854-b839-499106cfeb55 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:51.951422 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9e43ba3e-cd9f-4854-b839-499106cfeb55 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1215 microversion: 2.1 time: 0.509849 Sep 08 13:53:51.955106 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 25/48] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:53:51 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1215 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:51.977329 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8bf1587c-37ac-4f0f-aa23-b6c4ccd4b4c9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1525118878", "imageRef": "20b30425-92e5-4d78-8502-45557526ea49", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "a265cc60-700e-4951-9295-70d02eb6ee91"}], "security_groups": [{"name": "tempest-securitygroup--744451698"}], "key_name": "tempest-keypair-494104329", "user_data": "}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:53:52.223675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.network.neutron [None req-8bf1587c-37ac-4f0f-aa23-b6c4ccd4b4c9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] validate_networks() for [('a265cc60-700e-4951-9295-70d02eb6ee91', None, None, None, None, None)] {{(pid=127602) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 13:53:52.779672 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-205a8ad9-9269-41ee-b093-9e05561b4dca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Flavor limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 13:53:52.780692 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-205a8ad9-9269-41ee-b093-9e05561b4dca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Image limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 13:53:52.781272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-205a8ad9-9269-41ee-b093-9e05561b4dca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Flavor pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 13:53:52.782471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-205a8ad9-9269-41ee-b093-9e05561b4dca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Image pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 13:53:52.783122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-205a8ad9-9269-41ee-b093-9e05561b4dca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 13:53:52.800978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-205a8ad9-9269-41ee-b093-9e05561b4dca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:52.805752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-205a8ad9-9269-41ee-b093-9e05561b4dca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:52.806473 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-205a8ad9-9269-41ee-b093-9e05561b4dca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:52.807593 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-205a8ad9-9269-41ee-b093-9e05561b4dca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:52.808503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-205a8ad9-9269-41ee-b093-9e05561b4dca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:52.809204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-205a8ad9-9269-41ee-b093-9e05561b4dca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:52.877203 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-205a8ad9-9269-41ee-b093-9e05561b4dca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Getting quotas for project 1346d4105dce4c6e928fcb11cde56854. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:53:52.898045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-205a8ad9-9269-41ee-b093-9e05561b4dca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Getting quotas for user 792808dcbf894aa98c671cf43ac21b8b and project 1346d4105dce4c6e928fcb11cde56854. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:53:52.958893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-205a8ad9-9269-41ee-b093-9e05561b4dca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Going to run 1 instances... {{(pid=127601) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 13:53:52.966304 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-205a8ad9-9269-41ee-b093-9e05561b4dca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='20b30425-92e5-4d78-8502-45557526ea49',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127601) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 13:53:53.052117 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-8bf1587c-37ac-4f0f-aa23-b6c4ccd4b4c9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Flavor limits 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 13:53:53.060347 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-8bf1587c-37ac-4f0f-aa23-b6c4ccd4b4c9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Image limits 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 13:53:53.060347 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-8bf1587c-37ac-4f0f-aa23-b6c4ccd4b4c9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Flavor pref 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 13:53:53.060347 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-8bf1587c-37ac-4f0f-aa23-b6c4ccd4b4c9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Image pref 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 13:53:53.060347 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-8bf1587c-37ac-4f0f-aa23-b6c4ccd4b4c9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 13:53:53.070215 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8bf1587c-37ac-4f0f-aa23-b6c4ccd4b4c9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:53.071459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8bf1587c-37ac-4f0f-aa23-b6c4ccd4b4c9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:53.072320 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8bf1587c-37ac-4f0f-aa23-b6c4ccd4b4c9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:53.073676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8bf1587c-37ac-4f0f-aa23-b6c4ccd4b4c9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:53.074407 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8bf1587c-37ac-4f0f-aa23-b6c4ccd4b4c9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:53.075588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8bf1587c-37ac-4f0f-aa23-b6c4ccd4b4c9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:53.099297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-8bf1587c-37ac-4f0f-aa23-b6c4ccd4b4c9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Getting quotas for project a4e0fba6cab34c429c73fcfac7ebfd94. Resources: {'cores', 'instances', 'ram'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:53:53.103839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-8bf1587c-37ac-4f0f-aa23-b6c4ccd4b4c9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Getting quotas for user eca89f435299415c8448931e73f5b0fc and project a4e0fba6cab34c429c73fcfac7ebfd94. Resources: {'cores', 'instances', 'ram'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:53:53.120711 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8bf1587c-37ac-4f0f-aa23-b6c4ccd4b4c9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Going to run 1 instances... {{(pid=127602) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 13:53:53.124353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8bf1587c-37ac-4f0f-aa23-b6c4ccd4b4c9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='20b30425-92e5-4d78-8502-45557526ea49',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127602) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 13:53:53.176081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-205a8ad9-9269-41ee-b093-9e05561b4dca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.637959 Sep 08 13:53:53.177425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 24/49] 10.222.0.25 () {60 vars in 1147 bytes} [Thu Sep 8 13:53:51 2022] POST /compute/v2.1/servers => generated 383 bytes in 1640 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 08 13:53:53.188657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d035bfeb-f42b-4aef-8e6c-d7b273f2a765 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:53.190619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d035bfeb-f42b-4aef-8e6c-d7b273f2a765 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:53.228183 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d035bfeb-f42b-4aef-8e6c-d7b273f2a765 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:53.234950 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d035bfeb-f42b-4aef-8e6c-d7b273f2a765 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:53.237413 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d035bfeb-f42b-4aef-8e6c-d7b273f2a765 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:53.239696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d035bfeb-f42b-4aef-8e6c-d7b273f2a765 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:53.242980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d035bfeb-f42b-4aef-8e6c-d7b273f2a765 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:53.243781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d035bfeb-f42b-4aef-8e6c-d7b273f2a765 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:53.323468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8bf1587c-37ac-4f0f-aa23-b6c4ccd4b4c9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.359710 Sep 08 13:53:53.327172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 26/50] 10.222.0.25 () {60 vars in 1147 bytes} [Thu Sep 8 13:53:51 2022] POST /compute/v2.1/servers => generated 401 bytes in 1372 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 08 13:53:53.333690 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5c169fb1-ec95-40a6-830a-0d987383c490 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:53.371335 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5c169fb1-ec95-40a6-830a-0d987383c490 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:53.688210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5c169fb1-ec95-40a6-830a-0d987383c490 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:53.703119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5c169fb1-ec95-40a6-830a-0d987383c490 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1221 microversion: 2.1 time: 0.373496 Sep 08 13:53:53.742643 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 27/51] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:53:53 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1221 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:53.749467 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d035bfeb-f42b-4aef-8e6c-d7b273f2a765 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:53.764223 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7e29be36-1e93-4f48-bda5-55e5f2620bee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:53.775519 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7e29be36-1e93-4f48-bda5-55e5f2620bee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:53.823926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d035bfeb-f42b-4aef-8e6c-d7b273f2a765 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1211 microversion: 2.1 time: 0.645028 Sep 08 13:53:53.825727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 25/52] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:53:53 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1211 bytes in 647 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:53.909525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7e29be36-1e93-4f48-bda5-55e5f2620bee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:53.917984 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7e29be36-1e93-4f48-bda5-55e5f2620bee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1241 microversion: 2.1 time: 0.172393 Sep 08 13:53:53.919322 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 28/53] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:53:53 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1241 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:53.979713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0ac250b7-802a-4984-9fbc-4614d61a7b71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1996208931", "imageRef": "20b30425-92e5-4d78-8502-45557526ea49", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "5ee979e1-44f7-4b47-8a29-595077d970e8"}], "security_groups": [{"name": "tempest-securitygroup--1570497962"}], "key_name": "tempest-keypair-920229013", "user_data": "}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:53:54.240730 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.network.neutron [None req-0ac250b7-802a-4984-9fbc-4614d61a7b71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] validate_networks() for [('5ee979e1-44f7-4b47-8a29-595077d970e8', None, None, None, None, None)] {{(pid=127601) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 13:53:54.734621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c5f51106-87eb-4aae-a3dd-811f45e1ade1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:54.737937 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c5f51106-87eb-4aae-a3dd-811f45e1ade1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:54.757038 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5f51106-87eb-4aae-a3dd-811f45e1ade1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:54.769963 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5f51106-87eb-4aae-a3dd-811f45e1ade1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:54.780313 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5f51106-87eb-4aae-a3dd-811f45e1ade1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.019s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:54.781228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5f51106-87eb-4aae-a3dd-811f45e1ade1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:54.781721 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5f51106-87eb-4aae-a3dd-811f45e1ade1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:54.782178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5f51106-87eb-4aae-a3dd-811f45e1ade1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:54.894276 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-0ac250b7-802a-4984-9fbc-4614d61a7b71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Flavor limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 13:53:54.894753 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-0ac250b7-802a-4984-9fbc-4614d61a7b71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Image limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 13:53:54.895472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-0ac250b7-802a-4984-9fbc-4614d61a7b71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Flavor pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 13:53:54.895960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-0ac250b7-802a-4984-9fbc-4614d61a7b71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Image pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 13:53:54.896504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-0ac250b7-802a-4984-9fbc-4614d61a7b71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 13:53:54.919246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ac250b7-802a-4984-9fbc-4614d61a7b71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:54.919246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ac250b7-802a-4984-9fbc-4614d61a7b71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:54.921443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ac250b7-802a-4984-9fbc-4614d61a7b71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:54.923876 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ac250b7-802a-4984-9fbc-4614d61a7b71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:54.925008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ac250b7-802a-4984-9fbc-4614d61a7b71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:54.925953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ac250b7-802a-4984-9fbc-4614d61a7b71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:55.055478 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-0ac250b7-802a-4984-9fbc-4614d61a7b71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Getting quotas for project 4c194dd986d442d3bcaa83442a49ff70. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:53:55.076819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-0ac250b7-802a-4984-9fbc-4614d61a7b71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Getting quotas for user e9925313d1d446fa81a00cb55348dff7 and project 4c194dd986d442d3bcaa83442a49ff70. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:53:55.117017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0ac250b7-802a-4984-9fbc-4614d61a7b71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Going to run 1 instances... {{(pid=127601) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 13:53:55.122884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0ac250b7-802a-4984-9fbc-4614d61a7b71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='20b30425-92e5-4d78-8502-45557526ea49',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127601) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 13:53:55.210364 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c5f51106-87eb-4aae-a3dd-811f45e1ade1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:55.236037 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c5f51106-87eb-4aae-a3dd-811f45e1ade1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1225 microversion: 2.1 time: 0.508968 Sep 08 13:53:55.239892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 29/54] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:53:54 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1225 bytes in 513 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:55.257605 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4eba90a0-8960-461e-943e-50aa72708778 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:55.260120 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4eba90a0-8960-461e-943e-50aa72708778 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:55.305048 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4eba90a0-8960-461e-943e-50aa72708778 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:55.326818 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4eba90a0-8960-461e-943e-50aa72708778 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:55.327831 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4eba90a0-8960-461e-943e-50aa72708778 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.022s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:55.328629 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4eba90a0-8960-461e-943e-50aa72708778 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:55.329121 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4eba90a0-8960-461e-943e-50aa72708778 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:55.336083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4eba90a0-8960-461e-943e-50aa72708778 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:55.415153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0ac250b7-802a-4984-9fbc-4614d61a7b71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 1.449523 Sep 08 13:53:55.420319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 26/55] 10.222.0.25 () {62 vars in 1188 bytes} [Thu Sep 8 13:53:53 2022] POST /compute/v2.1/servers => generated 402 bytes in 1455 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 08 13:53:55.481647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1d8388ce-867b-4897-9b01-b0cf5bfc1b0d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:55.485716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1d8388ce-867b-4897-9b01-b0cf5bfc1b0d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:55.504484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d8388ce-867b-4897-9b01-b0cf5bfc1b0d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:55.512782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d8388ce-867b-4897-9b01-b0cf5bfc1b0d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:55.513484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d8388ce-867b-4897-9b01-b0cf5bfc1b0d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:55.514456 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d8388ce-867b-4897-9b01-b0cf5bfc1b0d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:55.517165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d8388ce-867b-4897-9b01-b0cf5bfc1b0d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:55.517834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d8388ce-867b-4897-9b01-b0cf5bfc1b0d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:55.580618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4eba90a0-8960-461e-943e-50aa72708778 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:55.593168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4eba90a0-8960-461e-943e-50aa72708778 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1275 microversion: 2.1 time: 0.350365 Sep 08 13:53:55.594438 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 30/56] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:53:55 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1275 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:55.600861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-faf84dec-23f1-48f0-8eb1-0292dc7e2426 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:55.601773 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-faf84dec-23f1-48f0-8eb1-0292dc7e2426 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:55.679361 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-faf84dec-23f1-48f0-8eb1-0292dc7e2426 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:55.687278 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1d8388ce-867b-4897-9b01-b0cf5bfc1b0d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:55.696882 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1d8388ce-867b-4897-9b01-b0cf5bfc1b0d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1245 microversion: 2.1 time: 0.275385 Sep 08 13:53:55.698469 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 27/57] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:53:55 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1245 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:55.701231 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-faf84dec-23f1-48f0-8eb1-0292dc7e2426 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:55.706058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-faf84dec-23f1-48f0-8eb1-0292dc7e2426 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:55.707468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-faf84dec-23f1-48f0-8eb1-0292dc7e2426 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1281 microversion: 2.20 time: 0.111631 Sep 08 13:53:55.708228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 31/58] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:53:55 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1281 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:53:56.265477 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ec496063-c175-4147-896f-6a047f6b634e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:56.266704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ec496063-c175-4147-896f-6a047f6b634e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:56.275990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec496063-c175-4147-896f-6a047f6b634e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:56.288452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec496063-c175-4147-896f-6a047f6b634e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:56.292912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec496063-c175-4147-896f-6a047f6b634e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:56.295455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec496063-c175-4147-896f-6a047f6b634e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:56.299809 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec496063-c175-4147-896f-6a047f6b634e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:56.301444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec496063-c175-4147-896f-6a047f6b634e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:56.620955 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-04499f5e-c32d-4501-8c15-390d13b358a4 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:56.623800 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-04499f5e-c32d-4501-8c15-390d13b358a4 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:56.641572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-04499f5e-c32d-4501-8c15-390d13b358a4 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:56.642078 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-04499f5e-c32d-4501-8c15-390d13b358a4 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:56.642549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-04499f5e-c32d-4501-8c15-390d13b358a4 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:56.647651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-04499f5e-c32d-4501-8c15-390d13b358a4 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:56.651535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-04499f5e-c32d-4501-8c15-390d13b358a4 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:56.652077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-04499f5e-c32d-4501-8c15-390d13b358a4 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:56.663997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ec496063-c175-4147-896f-6a047f6b634e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:56.686481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ec496063-c175-4147-896f-6a047f6b634e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1217 microversion: 2.1 time: 0.426265 Sep 08 13:53:56.708134 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 28/59] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:53:56 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1217 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:56.738401 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-89747e78-4e14-40e4-85ac-216261e7ab28 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:56.740726 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-89747e78-4e14-40e4-85ac-216261e7ab28 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:56.769906 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-89747e78-4e14-40e4-85ac-216261e7ab28 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:56.773131 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-89747e78-4e14-40e4-85ac-216261e7ab28 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:56.778437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-89747e78-4e14-40e4-85ac-216261e7ab28 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:56.781507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-89747e78-4e14-40e4-85ac-216261e7ab28 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:56.785934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-89747e78-4e14-40e4-85ac-216261e7ab28 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:56.786860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-89747e78-4e14-40e4-85ac-216261e7ab28 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:56.856728 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-04499f5e-c32d-4501-8c15-390d13b358a4 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:56.870015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-04499f5e-c32d-4501-8c15-390d13b358a4 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1273 microversion: 2.1 time: 0.253098 Sep 08 13:53:56.872176 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 32/60] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:53:56 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1273 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:56.877171 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d9161cc1-2e34-46a0-9c1b-1c5d0b6d44ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:56.903748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d9161cc1-2e34-46a0-9c1b-1c5d0b6d44ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:57.119883 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d9161cc1-2e34-46a0-9c1b-1c5d0b6d44ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:57.129348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d9161cc1-2e34-46a0-9c1b-1c5d0b6d44ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:57.130870 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d9161cc1-2e34-46a0-9c1b-1c5d0b6d44ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:57.131989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d9161cc1-2e34-46a0-9c1b-1c5d0b6d44ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1281 microversion: 2.20 time: 0.258594 Sep 08 13:53:57.133322 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 33/61] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:53:56 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1281 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:53:57.198924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-89747e78-4e14-40e4-85ac-216261e7ab28 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:57.210600 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-89747e78-4e14-40e4-85ac-216261e7ab28 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1237 microversion: 2.1 time: 0.485517 Sep 08 13:53:57.212920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 29/62] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:53:56 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1237 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:53:57.714568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-33154218-b629-4cc6-934a-4ba8108fa2df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:57.715266 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-33154218-b629-4cc6-934a-4ba8108fa2df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:57.721950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33154218-b629-4cc6-934a-4ba8108fa2df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:57.727360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33154218-b629-4cc6-934a-4ba8108fa2df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:57.730301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33154218-b629-4cc6-934a-4ba8108fa2df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:57.731786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33154218-b629-4cc6-934a-4ba8108fa2df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:57.737133 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33154218-b629-4cc6-934a-4ba8108fa2df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:57.737582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33154218-b629-4cc6-934a-4ba8108fa2df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:57.897565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-23867b23-408a-45bc-a01e-345741347792 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:57.899695 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-23867b23-408a-45bc-a01e-345741347792 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:57.910628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23867b23-408a-45bc-a01e-345741347792 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:57.912019 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23867b23-408a-45bc-a01e-345741347792 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:57.912497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23867b23-408a-45bc-a01e-345741347792 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:57.930107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23867b23-408a-45bc-a01e-345741347792 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:57.930829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23867b23-408a-45bc-a01e-345741347792 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.018s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:57.939074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23867b23-408a-45bc-a01e-345741347792 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:58.112435 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-33154218-b629-4cc6-934a-4ba8108fa2df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:58.135080 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-33154218-b629-4cc6-934a-4ba8108fa2df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1273 microversion: 2.1 time: 0.421046 Sep 08 13:53:58.137650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 34/63] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:53:57 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1273 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:58.170481 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bde1010e-296f-41f9-aa12-8e413992eebb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:58.175138 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bde1010e-296f-41f9-aa12-8e413992eebb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:58.209776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bde1010e-296f-41f9-aa12-8e413992eebb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:58.212733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bde1010e-296f-41f9-aa12-8e413992eebb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:58.212733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bde1010e-296f-41f9-aa12-8e413992eebb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:58.214054 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bde1010e-296f-41f9-aa12-8e413992eebb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:58.218626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bde1010e-296f-41f9-aa12-8e413992eebb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:58.223863 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bde1010e-296f-41f9-aa12-8e413992eebb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:58.533848 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-23867b23-408a-45bc-a01e-345741347792 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:58.546076 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-23867b23-408a-45bc-a01e-345741347792 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1315 microversion: 2.1 time: 0.654003 Sep 08 13:53:58.560604 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 30/64] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:53:57 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1315 bytes in 669 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:58.572040 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d68b4a12-cfe7-4087-afa4-edc2ea01e60b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:58.572724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d68b4a12-cfe7-4087-afa4-edc2ea01e60b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:58.589041 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d68b4a12-cfe7-4087-afa4-edc2ea01e60b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:58.589720 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d68b4a12-cfe7-4087-afa4-edc2ea01e60b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:58.595158 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d68b4a12-cfe7-4087-afa4-edc2ea01e60b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:58.595365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d68b4a12-cfe7-4087-afa4-edc2ea01e60b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:58.595730 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d68b4a12-cfe7-4087-afa4-edc2ea01e60b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:58.596172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d68b4a12-cfe7-4087-afa4-edc2ea01e60b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:58.597352 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-bde1010e-296f-41f9-aa12-8e413992eebb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:58.691053 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-bde1010e-296f-41f9-aa12-8e413992eebb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:58.691053 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-bde1010e-296f-41f9-aa12-8e413992eebb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:58.691053 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bde1010e-296f-41f9-aa12-8e413992eebb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1277 microversion: 2.20 time: 0.518235 Sep 08 13:53:58.692012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 35/65] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:53:58 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1277 bytes in 520 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:53:58.949270 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d68b4a12-cfe7-4087-afa4-edc2ea01e60b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:58.981240 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d68b4a12-cfe7-4087-afa4-edc2ea01e60b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1293 microversion: 2.1 time: 0.414351 Sep 08 13:53:58.985090 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 31/66] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:53:58 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1293 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:59.164909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-14e65fec-9a88-4310-9b72-a3fed4eb5165 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:59.166844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-14e65fec-9a88-4310-9b72-a3fed4eb5165 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:59.206949 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-14e65fec-9a88-4310-9b72-a3fed4eb5165 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:59.218387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-14e65fec-9a88-4310-9b72-a3fed4eb5165 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:59.223459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-14e65fec-9a88-4310-9b72-a3fed4eb5165 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:59.225554 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-14e65fec-9a88-4310-9b72-a3fed4eb5165 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:59.236466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-14e65fec-9a88-4310-9b72-a3fed4eb5165 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:59.236669 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-14e65fec-9a88-4310-9b72-a3fed4eb5165 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:59.573468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9bce5aba-4b37-4705-86ca-40f8c8adc8e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:59.578835 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9bce5aba-4b37-4705-86ca-40f8c8adc8e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:59.588712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bce5aba-4b37-4705-86ca-40f8c8adc8e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:59.591510 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bce5aba-4b37-4705-86ca-40f8c8adc8e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:59.592159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bce5aba-4b37-4705-86ca-40f8c8adc8e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:59.593056 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bce5aba-4b37-4705-86ca-40f8c8adc8e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:59.599840 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bce5aba-4b37-4705-86ca-40f8c8adc8e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:59.605011 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bce5aba-4b37-4705-86ca-40f8c8adc8e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:59.802760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-14e65fec-9a88-4310-9b72-a3fed4eb5165 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:53:59.811545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-14e65fec-9a88-4310-9b72-a3fed4eb5165 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1279 microversion: 2.1 time: 0.650653 Sep 08 13:53:59.812662 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 36/67] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:53:59 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1279 bytes in 652 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:53:59.842249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-396a191d-3321-461d-ad1e-4b4669830ef3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:53:59.842249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-396a191d-3321-461d-ad1e-4b4669830ef3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:53:59.861286 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-396a191d-3321-461d-ad1e-4b4669830ef3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:59.861992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-396a191d-3321-461d-ad1e-4b4669830ef3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:59.862567 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-396a191d-3321-461d-ad1e-4b4669830ef3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:53:59.864244 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-396a191d-3321-461d-ad1e-4b4669830ef3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:53:59.867751 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-396a191d-3321-461d-ad1e-4b4669830ef3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:53:59.869475 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-396a191d-3321-461d-ad1e-4b4669830ef3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:00.366030 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-396a191d-3321-461d-ad1e-4b4669830ef3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:00.378437 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-396a191d-3321-461d-ad1e-4b4669830ef3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:00.380235 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-396a191d-3321-461d-ad1e-4b4669830ef3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:00.381595 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-396a191d-3321-461d-ad1e-4b4669830ef3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1277 microversion: 2.20 time: 0.567739 Sep 08 13:54:00.394829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 37/68] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:53:59 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1277 bytes in 581 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:00.417754 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b360fecf-7ada-4642-84e2-187a59214678 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:00.417754 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b360fecf-7ada-4642-84e2-187a59214678 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:00.422882 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9bce5aba-4b37-4705-86ca-40f8c8adc8e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:00.427960 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b360fecf-7ada-4642-84e2-187a59214678 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:00.428675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b360fecf-7ada-4642-84e2-187a59214678 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:00.429482 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b360fecf-7ada-4642-84e2-187a59214678 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:00.430465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b360fecf-7ada-4642-84e2-187a59214678 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:00.431081 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b360fecf-7ada-4642-84e2-187a59214678 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:00.431643 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b360fecf-7ada-4642-84e2-187a59214678 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:00.461861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9bce5aba-4b37-4705-86ca-40f8c8adc8e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1315 microversion: 2.1 time: 0.892787 Sep 08 13:54:00.463849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 32/69] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:53:59 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1315 bytes in 896 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:00.594941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b360fecf-7ada-4642-84e2-187a59214678 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:00.603922 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b360fecf-7ada-4642-84e2-187a59214678 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1293 microversion: 2.1 time: 0.207883 Sep 08 13:54:00.605486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 38/70] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:00 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1293 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:00.837510 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6f69d023-1501-4d34-b958-dd64686429ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:00.838109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6f69d023-1501-4d34-b958-dd64686429ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:00.846998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f69d023-1501-4d34-b958-dd64686429ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:00.850188 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f69d023-1501-4d34-b958-dd64686429ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:00.850188 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f69d023-1501-4d34-b958-dd64686429ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:00.850188 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f69d023-1501-4d34-b958-dd64686429ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:00.850188 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f69d023-1501-4d34-b958-dd64686429ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:00.850781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f69d023-1501-4d34-b958-dd64686429ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:01.033020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6f69d023-1501-4d34-b958-dd64686429ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:01.041327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6f69d023-1501-4d34-b958-dd64686429ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.207577 Sep 08 13:54:01.041937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 33/71] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:00 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:01.417278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e19242fd-8c6a-41c9-8cb8-8a9a5a5ec4db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:01.417278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e19242fd-8c6a-41c9-8cb8-8a9a5a5ec4db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:01.420246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e19242fd-8c6a-41c9-8cb8-8a9a5a5ec4db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:01.421009 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e19242fd-8c6a-41c9-8cb8-8a9a5a5ec4db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:01.421697 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e19242fd-8c6a-41c9-8cb8-8a9a5a5ec4db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:01.422739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e19242fd-8c6a-41c9-8cb8-8a9a5a5ec4db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:01.423607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e19242fd-8c6a-41c9-8cb8-8a9a5a5ec4db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:01.424216 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e19242fd-8c6a-41c9-8cb8-8a9a5a5ec4db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:01.486169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-043311a9-5f59-4452-8f1c-48b10b362101 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:01.487176 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-043311a9-5f59-4452-8f1c-48b10b362101 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:01.494389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-043311a9-5f59-4452-8f1c-48b10b362101 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:01.494953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-043311a9-5f59-4452-8f1c-48b10b362101 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:01.495603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-043311a9-5f59-4452-8f1c-48b10b362101 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:01.496326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-043311a9-5f59-4452-8f1c-48b10b362101 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:01.496896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-043311a9-5f59-4452-8f1c-48b10b362101 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:01.497489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-043311a9-5f59-4452-8f1c-48b10b362101 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:01.516762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e19242fd-8c6a-41c9-8cb8-8a9a5a5ec4db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:01.547164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e19242fd-8c6a-41c9-8cb8-8a9a5a5ec4db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:01.550228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e19242fd-8c6a-41c9-8cb8-8a9a5a5ec4db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:01.551580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e19242fd-8c6a-41c9-8cb8-8a9a5a5ec4db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1277 microversion: 2.20 time: 0.144818 Sep 08 13:54:01.552150 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 39/72] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:01 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1277 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:01.632256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-225e6148-75ad-4b20-89f7-4dafaedd9e26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:01.634020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-225e6148-75ad-4b20-89f7-4dafaedd9e26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:01.654297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-225e6148-75ad-4b20-89f7-4dafaedd9e26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:01.654798 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-225e6148-75ad-4b20-89f7-4dafaedd9e26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:01.655230 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-225e6148-75ad-4b20-89f7-4dafaedd9e26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:01.655972 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-225e6148-75ad-4b20-89f7-4dafaedd9e26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:01.657044 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-225e6148-75ad-4b20-89f7-4dafaedd9e26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:01.657479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-225e6148-75ad-4b20-89f7-4dafaedd9e26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:01.774499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-225e6148-75ad-4b20-89f7-4dafaedd9e26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:01.783351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-225e6148-75ad-4b20-89f7-4dafaedd9e26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1293 microversion: 2.1 time: 0.157423 Sep 08 13:54:01.783999 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 40/73] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:01 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1293 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:01.797153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-043311a9-5f59-4452-8f1c-48b10b362101 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:01.797153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-043311a9-5f59-4452-8f1c-48b10b362101 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1315 microversion: 2.1 time: 0.314207 Sep 08 13:54:01.797153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 34/74] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:01 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1315 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:01.914483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-f1e9f9bc-612d-4ef7-b494-5ed79340b9bb req-21ab4c44-04cc-4b5a-a74b-8ca781f1464a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "358e292a-9e96-4380-8ea5-3ed00372b4c3", "tag": "a0d989dd-29c8-414f-a850-15a4e3aea7a8"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:54:01.927682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-f1e9f9bc-612d-4ef7-b494-5ed79340b9bb req-21ab4c44-04cc-4b5a-a74b-8ca781f1464a service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:01.928152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-f1e9f9bc-612d-4ef7-b494-5ed79340b9bb req-21ab4c44-04cc-4b5a-a74b-8ca781f1464a service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:01.928578 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-f1e9f9bc-612d-4ef7-b494-5ed79340b9bb req-21ab4c44-04cc-4b5a-a74b-8ca781f1464a service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:01.947709 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-f1e9f9bc-612d-4ef7-b494-5ed79340b9bb req-21ab4c44-04cc-4b5a-a74b-8ca781f1464a service nova] Creating event network-changed:a0d989dd-29c8-414f-a850-15a4e3aea7a8 for instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 on hv-cinder-85646 Sep 08 13:54:01.948150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-f1e9f9bc-612d-4ef7-b494-5ed79340b9bb req-21ab4c44-04cc-4b5a-a74b-8ca781f1464a service nova] Acquiring lock "compute-rpcapi-router" {{(pid=127601) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 08 13:54:01.948533 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-f1e9f9bc-612d-4ef7-b494-5ed79340b9bb req-21ab4c44-04cc-4b5a-a74b-8ca781f1464a service nova] Acquired lock "compute-rpcapi-router" {{(pid=127601) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 08 13:54:01.949645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c2b3639-ad7d-49ed-bc10-6b6a4d1c2fd9 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:01.950038 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c2b3639-ad7d-49ed-bc10-6b6a4d1c2fd9 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:01.950480 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c2b3639-ad7d-49ed-bc10-6b6a4d1c2fd9 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:01.951196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c2b3639-ad7d-49ed-bc10-6b6a4d1c2fd9 None None] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:01.951647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c2b3639-ad7d-49ed-bc10-6b6a4d1c2fd9 None None] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:01.952062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c2b3639-ad7d-49ed-bc10-6b6a4d1c2fd9 None None] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:01.961979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.compute.rpcapi [None req-6c2b3639-ad7d-49ed-bc10-6b6a4d1c2fd9 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 08 13:54:01.962965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c2b3639-ad7d-49ed-bc10-6b6a4d1c2fd9 None None] Releasing lock "compute-rpcapi-router" {{(pid=127601) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 08 13:54:01.986123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6c2b3639-ad7d-49ed-bc10-6b6a4d1c2fd9 None None] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.188916 Sep 08 13:54:01.986965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 35/75] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:54:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:54:02.064818 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-43af2c19-7e7c-4cff-b32e-5b53bb63b408 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:02.065472 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-43af2c19-7e7c-4cff-b32e-5b53bb63b408 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:02.073872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=8 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:02.073872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-43af2c19-7e7c-4cff-b32e-5b53bb63b408 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:02.073872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-43af2c19-7e7c-4cff-b32e-5b53bb63b408 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:02.073872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-43af2c19-7e7c-4cff-b32e-5b53bb63b408 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:02.074336 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-43af2c19-7e7c-4cff-b32e-5b53bb63b408 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:02.074789 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-43af2c19-7e7c-4cff-b32e-5b53bb63b408 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:02.075207 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-43af2c19-7e7c-4cff-b32e-5b53bb63b408 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:02.296738 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-43af2c19-7e7c-4cff-b32e-5b53bb63b408 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:02.304463 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-43af2c19-7e7c-4cff-b32e-5b53bb63b408 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.243567 Sep 08 13:54:02.310543 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 41/76] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:02 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:02.602241 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-62fa5971-e5c2-4bad-8c17-7562d9f7fa0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:02.603665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-62fa5971-e5c2-4bad-8c17-7562d9f7fa0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:02.613228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-62fa5971-e5c2-4bad-8c17-7562d9f7fa0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:02.613965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-62fa5971-e5c2-4bad-8c17-7562d9f7fa0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:02.615123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-62fa5971-e5c2-4bad-8c17-7562d9f7fa0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:02.615987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-62fa5971-e5c2-4bad-8c17-7562d9f7fa0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:02.616584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-62fa5971-e5c2-4bad-8c17-7562d9f7fa0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:02.617165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-62fa5971-e5c2-4bad-8c17-7562d9f7fa0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:02.689827 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-62fa5971-e5c2-4bad-8c17-7562d9f7fa0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:02.698215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-62fa5971-e5c2-4bad-8c17-7562d9f7fa0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:02.700564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-62fa5971-e5c2-4bad-8c17-7562d9f7fa0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:02.706381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-62fa5971-e5c2-4bad-8c17-7562d9f7fa0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1277 microversion: 2.20 time: 0.105195 Sep 08 13:54:02.707549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 36/77] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:02 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1277 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:02.808502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-930e4696-de70-4937-b5f4-b445bb2543fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:02.816573 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-930e4696-de70-4937-b5f4-b445bb2543fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:02.819290 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-930e4696-de70-4937-b5f4-b445bb2543fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:02.821267 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-930e4696-de70-4937-b5f4-b445bb2543fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:02.822547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-930e4696-de70-4937-b5f4-b445bb2543fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:02.824825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-930e4696-de70-4937-b5f4-b445bb2543fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:02.826117 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-930e4696-de70-4937-b5f4-b445bb2543fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:02.826907 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-930e4696-de70-4937-b5f4-b445bb2543fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:02.832552 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-32e707cb-a2f6-40ad-bc93-1b1f5c2c7f32 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:02.833285 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-32e707cb-a2f6-40ad-bc93-1b1f5c2c7f32 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:02.849828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32e707cb-a2f6-40ad-bc93-1b1f5c2c7f32 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:02.850559 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32e707cb-a2f6-40ad-bc93-1b1f5c2c7f32 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:02.851296 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32e707cb-a2f6-40ad-bc93-1b1f5c2c7f32 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:02.852011 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32e707cb-a2f6-40ad-bc93-1b1f5c2c7f32 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:02.852480 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32e707cb-a2f6-40ad-bc93-1b1f5c2c7f32 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:02.852890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32e707cb-a2f6-40ad-bc93-1b1f5c2c7f32 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:02.949189 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-930e4696-de70-4937-b5f4-b445bb2543fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:02.958878 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-930e4696-de70-4937-b5f4-b445bb2543fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1301 microversion: 2.1 time: 0.154270 Sep 08 13:54:02.975682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 42/78] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:02 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1301 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:03.068754 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-32e707cb-a2f6-40ad-bc93-1b1f5c2c7f32 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:03.094780 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-32e707cb-a2f6-40ad-bc93-1b1f5c2c7f32 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1315 microversion: 2.1 time: 0.279916 Sep 08 13:54:03.107161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 37/79] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:02 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1315 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:03.337979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c19752b2-fce9-4697-8870-a9814f006b4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:03.338904 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c19752b2-fce9-4697-8870-a9814f006b4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:03.344854 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=4 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:03.371156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c19752b2-fce9-4697-8870-a9814f006b4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:03.372757 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c19752b2-fce9-4697-8870-a9814f006b4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:03.373579 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c19752b2-fce9-4697-8870-a9814f006b4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:03.374541 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c19752b2-fce9-4697-8870-a9814f006b4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:03.375772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c19752b2-fce9-4697-8870-a9814f006b4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:03.377926 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c19752b2-fce9-4697-8870-a9814f006b4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:03.582675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c19752b2-fce9-4697-8870-a9814f006b4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:03.602208 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c19752b2-fce9-4697-8870-a9814f006b4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.269993 Sep 08 13:54:03.604218 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 43/80] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:03 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:03.739148 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3cfc244a-5beb-43cc-b993-2a9b8cba274f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:03.739760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3cfc244a-5beb-43cc-b993-2a9b8cba274f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:03.746786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3cfc244a-5beb-43cc-b993-2a9b8cba274f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:03.747361 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3cfc244a-5beb-43cc-b993-2a9b8cba274f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:03.748027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3cfc244a-5beb-43cc-b993-2a9b8cba274f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:03.748825 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3cfc244a-5beb-43cc-b993-2a9b8cba274f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:03.749348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3cfc244a-5beb-43cc-b993-2a9b8cba274f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:03.749864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3cfc244a-5beb-43cc-b993-2a9b8cba274f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:03.959945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3cfc244a-5beb-43cc-b993-2a9b8cba274f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:03.959945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3cfc244a-5beb-43cc-b993-2a9b8cba274f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:03.960592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3cfc244a-5beb-43cc-b993-2a9b8cba274f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:03.960592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3cfc244a-5beb-43cc-b993-2a9b8cba274f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1277 microversion: 2.20 time: 0.124378 Sep 08 13:54:03.960592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 38/81] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:03 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1277 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:03.983014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-dacd6d31-d6f3-4741-b0c5-033498ecd48a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:03.984409 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-dacd6d31-d6f3-4741-b0c5-033498ecd48a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:04.003078 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dacd6d31-d6f3-4741-b0c5-033498ecd48a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:04.003584 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dacd6d31-d6f3-4741-b0c5-033498ecd48a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:04.004013 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dacd6d31-d6f3-4741-b0c5-033498ecd48a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:04.005323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dacd6d31-d6f3-4741-b0c5-033498ecd48a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:04.005323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dacd6d31-d6f3-4741-b0c5-033498ecd48a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:04.005612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dacd6d31-d6f3-4741-b0c5-033498ecd48a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:04.085845 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-dacd6d31-d6f3-4741-b0c5-033498ecd48a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:04.104077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-dacd6d31-d6f3-4741-b0c5-033498ecd48a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1301 microversion: 2.1 time: 0.125226 Sep 08 13:54:04.112428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 44/82] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:03 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1301 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:04.121326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1adb2f8b-2b3e-420a-bf46-acf2d53b0fa1 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:04.122759 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1adb2f8b-2b3e-420a-bf46-acf2d53b0fa1 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:04.136949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1adb2f8b-2b3e-420a-bf46-acf2d53b0fa1 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:04.136949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1adb2f8b-2b3e-420a-bf46-acf2d53b0fa1 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:04.136949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1adb2f8b-2b3e-420a-bf46-acf2d53b0fa1 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:04.136949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1adb2f8b-2b3e-420a-bf46-acf2d53b0fa1 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:04.137637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1adb2f8b-2b3e-420a-bf46-acf2d53b0fa1 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:04.137637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1adb2f8b-2b3e-420a-bf46-acf2d53b0fa1 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:04.427061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1adb2f8b-2b3e-420a-bf46-acf2d53b0fa1 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:04.427061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1adb2f8b-2b3e-420a-bf46-acf2d53b0fa1 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1315 microversion: 2.1 time: 0.287506 Sep 08 13:54:04.427061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 39/83] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:04 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1315 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:04.648848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-89fd0067-dd2b-4e2c-bc1f-6956c29e5867 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:04.655308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-89fd0067-dd2b-4e2c-bc1f-6956c29e5867 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:04.656162 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89fd0067-dd2b-4e2c-bc1f-6956c29e5867 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:04.656657 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89fd0067-dd2b-4e2c-bc1f-6956c29e5867 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:04.657745 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89fd0067-dd2b-4e2c-bc1f-6956c29e5867 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:04.658976 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89fd0067-dd2b-4e2c-bc1f-6956c29e5867 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:04.659617 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89fd0067-dd2b-4e2c-bc1f-6956c29e5867 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:04.660053 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89fd0067-dd2b-4e2c-bc1f-6956c29e5867 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:04.868176 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-89fd0067-dd2b-4e2c-bc1f-6956c29e5867 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:04.886552 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-89fd0067-dd2b-4e2c-bc1f-6956c29e5867 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.243526 Sep 08 13:54:05.001312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 45/84] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:04 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:05.002451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2b01a8e6-8c9e-4564-a1e7-840e0cd8255d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:05.002451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2b01a8e6-8c9e-4564-a1e7-840e0cd8255d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:05.002451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b01a8e6-8c9e-4564-a1e7-840e0cd8255d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:05.002451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b01a8e6-8c9e-4564-a1e7-840e0cd8255d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:05.003194 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b01a8e6-8c9e-4564-a1e7-840e0cd8255d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:05.003194 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b01a8e6-8c9e-4564-a1e7-840e0cd8255d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:05.003194 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b01a8e6-8c9e-4564-a1e7-840e0cd8255d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:05.003194 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b01a8e6-8c9e-4564-a1e7-840e0cd8255d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:05.003617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2b01a8e6-8c9e-4564-a1e7-840e0cd8255d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:05.003617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2b01a8e6-8c9e-4564-a1e7-840e0cd8255d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:05.003860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2b01a8e6-8c9e-4564-a1e7-840e0cd8255d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:05.003860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2b01a8e6-8c9e-4564-a1e7-840e0cd8255d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1277 microversion: 2.20 time: 0.099977 Sep 08 13:54:05.003860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 40/85] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:04 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1277 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:54:05.129838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7e2cb2ce-e4cd-4626-ba44-fd9232516154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:05.130962 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7e2cb2ce-e4cd-4626-ba44-fd9232516154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:05.145221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e2cb2ce-e4cd-4626-ba44-fd9232516154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:05.146928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e2cb2ce-e4cd-4626-ba44-fd9232516154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:05.149753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e2cb2ce-e4cd-4626-ba44-fd9232516154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:05.151273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e2cb2ce-e4cd-4626-ba44-fd9232516154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:05.151899 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e2cb2ce-e4cd-4626-ba44-fd9232516154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:05.152547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e2cb2ce-e4cd-4626-ba44-fd9232516154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:05.244811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7e2cb2ce-e4cd-4626-ba44-fd9232516154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:05.253982 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:05.258056 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7e2cb2ce-e4cd-4626-ba44-fd9232516154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1301 microversion: 2.1 time: 0.132114 Sep 08 13:54:05.259522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 46/86] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:05 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1301 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:05.436418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a84cdd59-352b-48e8-a447-07744a6bba37 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:05.437173 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a84cdd59-352b-48e8-a447-07744a6bba37 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:05.440988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=24 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:05.446598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a84cdd59-352b-48e8-a447-07744a6bba37 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:05.447288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a84cdd59-352b-48e8-a447-07744a6bba37 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:05.447871 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a84cdd59-352b-48e8-a447-07744a6bba37 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:05.448731 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a84cdd59-352b-48e8-a447-07744a6bba37 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:05.449370 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a84cdd59-352b-48e8-a447-07744a6bba37 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:05.450011 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a84cdd59-352b-48e8-a447-07744a6bba37 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:05.631397 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a84cdd59-352b-48e8-a447-07744a6bba37 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:05.640994 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:05.643486 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a84cdd59-352b-48e8-a447-07744a6bba37 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1315 microversion: 2.1 time: 0.212345 Sep 08 13:54:05.644058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 41/87] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:05 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1315 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:06.010953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-08841371-5443-4ca9-bccc-948eefd5afb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:06.011642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2d6b74a8-e36c-421f-aff5-23980b3cd72e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:06.012055 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2d6b74a8-e36c-421f-aff5-23980b3cd72e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:06.013708 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-08841371-5443-4ca9-bccc-948eefd5afb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:06.024668 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2d6b74a8-e36c-421f-aff5-23980b3cd72e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:06.025155 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2d6b74a8-e36c-421f-aff5-23980b3cd72e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:06.025590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2d6b74a8-e36c-421f-aff5-23980b3cd72e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:06.026319 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2d6b74a8-e36c-421f-aff5-23980b3cd72e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:06.026586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08841371-5443-4ca9-bccc-948eefd5afb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:06.026761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2d6b74a8-e36c-421f-aff5-23980b3cd72e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:06.027082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08841371-5443-4ca9-bccc-948eefd5afb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:06.027681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08841371-5443-4ca9-bccc-948eefd5afb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:06.029160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08841371-5443-4ca9-bccc-948eefd5afb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:06.029849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08841371-5443-4ca9-bccc-948eefd5afb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:06.030960 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2d6b74a8-e36c-421f-aff5-23980b3cd72e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:06.032681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08841371-5443-4ca9-bccc-948eefd5afb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:06.241470 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-08841371-5443-4ca9-bccc-948eefd5afb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:06.241470 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-08841371-5443-4ca9-bccc-948eefd5afb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:06.243161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-08841371-5443-4ca9-bccc-948eefd5afb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:06.243161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-08841371-5443-4ca9-bccc-948eefd5afb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1277 microversion: 2.20 time: 0.197306 Sep 08 13:54:06.243161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 42/88] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:06 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1277 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:54:06.282893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5a2e156a-ec1b-448f-a9f4-d8af25781a79 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:06.283518 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5a2e156a-ec1b-448f-a9f4-d8af25781a79 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:06.291515 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-2d6b74a8-e36c-421f-aff5-23980b3cd72e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:06.292724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a2e156a-ec1b-448f-a9f4-d8af25781a79 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:06.293272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a2e156a-ec1b-448f-a9f4-d8af25781a79 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:06.293748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a2e156a-ec1b-448f-a9f4-d8af25781a79 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:06.294492 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a2e156a-ec1b-448f-a9f4-d8af25781a79 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:06.298592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a2e156a-ec1b-448f-a9f4-d8af25781a79 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:06.299262 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a2e156a-ec1b-448f-a9f4-d8af25781a79 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:06.303410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2d6b74a8-e36c-421f-aff5-23980b3cd72e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.297840 Sep 08 13:54:06.304346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 47/89] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:06 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:06.405038 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5a2e156a-ec1b-448f-a9f4-d8af25781a79 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:06.413363 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5a2e156a-ec1b-448f-a9f4-d8af25781a79 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1301 microversion: 2.1 time: 0.134651 Sep 08 13:54:06.416281 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 43/90] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:06 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1301 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:06.668512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7431c724-3f13-41d4-8566-ac19823176c2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:06.670485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7431c724-3f13-41d4-8566-ac19823176c2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:06.678853 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7431c724-3f13-41d4-8566-ac19823176c2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:06.679345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7431c724-3f13-41d4-8566-ac19823176c2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:06.680159 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7431c724-3f13-41d4-8566-ac19823176c2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:06.681035 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7431c724-3f13-41d4-8566-ac19823176c2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:06.681492 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7431c724-3f13-41d4-8566-ac19823176c2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:06.681922 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7431c724-3f13-41d4-8566-ac19823176c2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:06.838925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7431c724-3f13-41d4-8566-ac19823176c2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:06.843559 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7431c724-3f13-41d4-8566-ac19823176c2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1315 microversion: 2.1 time: 0.178917 Sep 08 13:54:06.844132 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 48/91] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:06 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1315 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:07.265721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-26da9400-c3ed-4d2b-aaf2-d313c6d34d0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:07.265721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-26da9400-c3ed-4d2b-aaf2-d313c6d34d0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:07.265721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=9 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:07.265721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26da9400-c3ed-4d2b-aaf2-d313c6d34d0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:07.267066 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26da9400-c3ed-4d2b-aaf2-d313c6d34d0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:07.267066 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26da9400-c3ed-4d2b-aaf2-d313c6d34d0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:07.267066 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26da9400-c3ed-4d2b-aaf2-d313c6d34d0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:07.267066 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26da9400-c3ed-4d2b-aaf2-d313c6d34d0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:07.267382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26da9400-c3ed-4d2b-aaf2-d313c6d34d0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:07.322973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-26da9400-c3ed-4d2b-aaf2-d313c6d34d0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:07.328493 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-55276e6f-a1ce-48f8-858d-68495b7ea14a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:07.329116 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-55276e6f-a1ce-48f8-858d-68495b7ea14a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:07.333404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=30 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:07.334370 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-26da9400-c3ed-4d2b-aaf2-d313c6d34d0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:07.336846 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-26da9400-c3ed-4d2b-aaf2-d313c6d34d0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:07.345501 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-26da9400-c3ed-4d2b-aaf2-d313c6d34d0e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1277 microversion: 2.20 time: 0.105681 Sep 08 13:54:07.346778 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 44/92] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:07 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1277 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:07.347365 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-55276e6f-a1ce-48f8-858d-68495b7ea14a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:07.348297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-55276e6f-a1ce-48f8-858d-68495b7ea14a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:07.348819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-55276e6f-a1ce-48f8-858d-68495b7ea14a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:07.349614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-55276e6f-a1ce-48f8-858d-68495b7ea14a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:07.350186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-55276e6f-a1ce-48f8-858d-68495b7ea14a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:07.350657 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-55276e6f-a1ce-48f8-858d-68495b7ea14a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:07.436315 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fab11711-2b93-4217-b11d-e2ac2473f042 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:07.437259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fab11711-2b93-4217-b11d-e2ac2473f042 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:07.443678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fab11711-2b93-4217-b11d-e2ac2473f042 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:07.444494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fab11711-2b93-4217-b11d-e2ac2473f042 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:07.445106 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fab11711-2b93-4217-b11d-e2ac2473f042 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:07.453610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fab11711-2b93-4217-b11d-e2ac2473f042 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:07.454383 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fab11711-2b93-4217-b11d-e2ac2473f042 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:07.459057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fab11711-2b93-4217-b11d-e2ac2473f042 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:07.536974 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fab11711-2b93-4217-b11d-e2ac2473f042 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:07.554383 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-55276e6f-a1ce-48f8-858d-68495b7ea14a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:07.562430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fab11711-2b93-4217-b11d-e2ac2473f042 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1301 microversion: 2.1 time: 0.122044 Sep 08 13:54:07.562430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 45/93] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:07 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1301 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:07.565188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-55276e6f-a1ce-48f8-858d-68495b7ea14a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.241112 Sep 08 13:54:07.565768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 49/94] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:07 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:07.867798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d01c8e68-01a8-4b81-8f71-b578b14888e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:07.868696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d01c8e68-01a8-4b81-8f71-b578b14888e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:07.881189 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d01c8e68-01a8-4b81-8f71-b578b14888e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:07.884226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d01c8e68-01a8-4b81-8f71-b578b14888e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:07.887960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d01c8e68-01a8-4b81-8f71-b578b14888e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:07.893134 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d01c8e68-01a8-4b81-8f71-b578b14888e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:07.893134 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d01c8e68-01a8-4b81-8f71-b578b14888e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:07.893134 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d01c8e68-01a8-4b81-8f71-b578b14888e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:08.056288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d01c8e68-01a8-4b81-8f71-b578b14888e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:08.065761 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d01c8e68-01a8-4b81-8f71-b578b14888e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1315 microversion: 2.1 time: 0.201370 Sep 08 13:54:08.066461 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 46/95] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:07 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1315 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:08.375101 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a0c33d39-8913-44ae-b736-e88a02041814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:08.375777 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a0c33d39-8913-44ae-b736-e88a02041814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:08.381732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0c33d39-8913-44ae-b736-e88a02041814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:08.382311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0c33d39-8913-44ae-b736-e88a02041814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:08.382653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0c33d39-8913-44ae-b736-e88a02041814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:08.383343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0c33d39-8913-44ae-b736-e88a02041814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:08.383785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0c33d39-8913-44ae-b736-e88a02041814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:08.384200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0c33d39-8913-44ae-b736-e88a02041814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:08.462159 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a0c33d39-8913-44ae-b736-e88a02041814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:08.476330 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a0c33d39-8913-44ae-b736-e88a02041814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:08.478943 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a0c33d39-8913-44ae-b736-e88a02041814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:08.480239 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a0c33d39-8913-44ae-b736-e88a02041814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1277 microversion: 2.20 time: 0.109108 Sep 08 13:54:08.480805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 50/96] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:08 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1277 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:08.580910 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6cf6f846-942e-482e-8cf5-918d84ec2d26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:08.582235 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6cf6f846-942e-482e-8cf5-918d84ec2d26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:08.589590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-35845380-0499-4a84-950c-97d584b6459a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:08.590627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-35845380-0499-4a84-950c-97d584b6459a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:08.602619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-35845380-0499-4a84-950c-97d584b6459a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:08.603340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-35845380-0499-4a84-950c-97d584b6459a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:08.603475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6cf6f846-942e-482e-8cf5-918d84ec2d26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:08.603934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6cf6f846-942e-482e-8cf5-918d84ec2d26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:08.604356 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6cf6f846-942e-482e-8cf5-918d84ec2d26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:08.604661 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-35845380-0499-4a84-950c-97d584b6459a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:08.605142 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6cf6f846-942e-482e-8cf5-918d84ec2d26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:08.605605 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6cf6f846-942e-482e-8cf5-918d84ec2d26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:08.606301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6cf6f846-942e-482e-8cf5-918d84ec2d26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:08.606548 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-35845380-0499-4a84-950c-97d584b6459a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:08.607280 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-35845380-0499-4a84-950c-97d584b6459a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:08.607877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-35845380-0499-4a84-950c-97d584b6459a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:08.725870 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6cf6f846-942e-482e-8cf5-918d84ec2d26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:08.736261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6cf6f846-942e-482e-8cf5-918d84ec2d26 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1301 microversion: 2.1 time: 0.162543 Sep 08 13:54:08.745185 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 47/97] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:08 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1301 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:08.801876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-35845380-0499-4a84-950c-97d584b6459a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:08.813277 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-35845380-0499-4a84-950c-97d584b6459a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.227784 Sep 08 13:54:08.813897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 51/98] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:08 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:09.088282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2fb4e5ea-05aa-4720-9f1e-c48e67fb4cbe tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:09.088884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2fb4e5ea-05aa-4720-9f1e-c48e67fb4cbe tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:09.095718 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fb4e5ea-05aa-4720-9f1e-c48e67fb4cbe tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:09.096188 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fb4e5ea-05aa-4720-9f1e-c48e67fb4cbe tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:09.096652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fb4e5ea-05aa-4720-9f1e-c48e67fb4cbe tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:09.097362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fb4e5ea-05aa-4720-9f1e-c48e67fb4cbe tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:09.097814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fb4e5ea-05aa-4720-9f1e-c48e67fb4cbe tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:09.098526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fb4e5ea-05aa-4720-9f1e-c48e67fb4cbe tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:09.235543 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2fb4e5ea-05aa-4720-9f1e-c48e67fb4cbe tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:09.243690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2fb4e5ea-05aa-4720-9f1e-c48e67fb4cbe tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1315 microversion: 2.1 time: 0.159153 Sep 08 13:54:09.244332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 48/99] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:09 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1315 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:09.522128 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-77ec6c05-6bac-4fc1-af74-1659b9c317bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:09.522128 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-77ec6c05-6bac-4fc1-af74-1659b9c317bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:09.530454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77ec6c05-6bac-4fc1-af74-1659b9c317bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:09.530454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77ec6c05-6bac-4fc1-af74-1659b9c317bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:09.531012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77ec6c05-6bac-4fc1-af74-1659b9c317bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:09.531712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77ec6c05-6bac-4fc1-af74-1659b9c317bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:09.532110 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77ec6c05-6bac-4fc1-af74-1659b9c317bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:09.532597 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77ec6c05-6bac-4fc1-af74-1659b9c317bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:09.605614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-77ec6c05-6bac-4fc1-af74-1659b9c317bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:09.615172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-77ec6c05-6bac-4fc1-af74-1659b9c317bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:09.616668 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-77ec6c05-6bac-4fc1-af74-1659b9c317bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:09.617818 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-77ec6c05-6bac-4fc1-af74-1659b9c317bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1277 microversion: 2.20 time: 0.100704 Sep 08 13:54:09.618636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 52/100] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:09 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1277 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:09.761322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4367e345-0013-4e07-a3cf-fd2cc905125c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:09.761935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4367e345-0013-4e07-a3cf-fd2cc905125c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:09.770169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4367e345-0013-4e07-a3cf-fd2cc905125c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:09.770705 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4367e345-0013-4e07-a3cf-fd2cc905125c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:09.771153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4367e345-0013-4e07-a3cf-fd2cc905125c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:09.771825 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4367e345-0013-4e07-a3cf-fd2cc905125c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:09.772514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4367e345-0013-4e07-a3cf-fd2cc905125c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:09.772680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4367e345-0013-4e07-a3cf-fd2cc905125c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:09.838443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-77bccd9f-88ec-4566-8452-547255535cad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:09.839062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-77bccd9f-88ec-4566-8452-547255535cad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:09.848847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77bccd9f-88ec-4566-8452-547255535cad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:09.850182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77bccd9f-88ec-4566-8452-547255535cad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:09.850779 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77bccd9f-88ec-4566-8452-547255535cad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:09.853601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77bccd9f-88ec-4566-8452-547255535cad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:09.853601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77bccd9f-88ec-4566-8452-547255535cad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:09.853601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77bccd9f-88ec-4566-8452-547255535cad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:09.894043 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4367e345-0013-4e07-a3cf-fd2cc905125c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:09.924494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4367e345-0013-4e07-a3cf-fd2cc905125c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1301 microversion: 2.1 time: 0.160876 Sep 08 13:54:09.936514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 49/101] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:09 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1301 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:10.269794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-20153315-7975-439d-9b85-e06484b93257 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:10.271161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-20153315-7975-439d-9b85-e06484b93257 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:10.284127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-20153315-7975-439d-9b85-e06484b93257 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:10.284923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-20153315-7975-439d-9b85-e06484b93257 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:10.285599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-20153315-7975-439d-9b85-e06484b93257 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:10.286589 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-20153315-7975-439d-9b85-e06484b93257 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:10.287249 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-20153315-7975-439d-9b85-e06484b93257 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:10.287869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-20153315-7975-439d-9b85-e06484b93257 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:10.350110 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-77bccd9f-88ec-4566-8452-547255535cad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:10.361906 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-77bccd9f-88ec-4566-8452-547255535cad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.527006 Sep 08 13:54:10.363628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 53/102] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:09 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:10.580729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-20153315-7975-439d-9b85-e06484b93257 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:10.588489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-20153315-7975-439d-9b85-e06484b93257 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1315 microversion: 2.1 time: 0.324271 Sep 08 13:54:10.589059 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 50/103] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:10 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1315 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:10.652905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cb6efba6-24b2-4df7-8738-375aff8a6bd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:10.654125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cb6efba6-24b2-4df7-8738-375aff8a6bd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:10.660597 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb6efba6-24b2-4df7-8738-375aff8a6bd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:10.661331 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb6efba6-24b2-4df7-8738-375aff8a6bd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:10.661967 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb6efba6-24b2-4df7-8738-375aff8a6bd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:10.662875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb6efba6-24b2-4df7-8738-375aff8a6bd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:10.663482 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb6efba6-24b2-4df7-8738-375aff8a6bd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:10.664057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb6efba6-24b2-4df7-8738-375aff8a6bd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:10.948763 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-99291349-7ad9-4310-aafe-b1678802d017 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:10.948763 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-99291349-7ad9-4310-aafe-b1678802d017 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:10.958264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-99291349-7ad9-4310-aafe-b1678802d017 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:10.963256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-99291349-7ad9-4310-aafe-b1678802d017 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:10.963972 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-99291349-7ad9-4310-aafe-b1678802d017 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:10.965032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-99291349-7ad9-4310-aafe-b1678802d017 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:10.966489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-99291349-7ad9-4310-aafe-b1678802d017 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:10.967539 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-99291349-7ad9-4310-aafe-b1678802d017 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:10.977012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-cb6efba6-24b2-4df7-8738-375aff8a6bd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:11.003979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-cb6efba6-24b2-4df7-8738-375aff8a6bd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:11.009528 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-cb6efba6-24b2-4df7-8738-375aff8a6bd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:11.015392 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cb6efba6-24b2-4df7-8738-375aff8a6bd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1333 microversion: 2.20 time: 0.375102 Sep 08 13:54:11.022803 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 54/104] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:10 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1333 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:11.181969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-99291349-7ad9-4310-aafe-b1678802d017 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:11.191619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-99291349-7ad9-4310-aafe-b1678802d017 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1299 microversion: 2.1 time: 0.250865 Sep 08 13:54:11.192209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 51/105] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:10 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1299 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:11.389134 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e9b60d6a-c27e-4d88-8df7-36585d72b57f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:11.389730 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e9b60d6a-c27e-4d88-8df7-36585d72b57f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:11.401423 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e9b60d6a-c27e-4d88-8df7-36585d72b57f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:11.408075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e9b60d6a-c27e-4d88-8df7-36585d72b57f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:11.408075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e9b60d6a-c27e-4d88-8df7-36585d72b57f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:11.420984 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e9b60d6a-c27e-4d88-8df7-36585d72b57f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:11.422839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e9b60d6a-c27e-4d88-8df7-36585d72b57f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:11.424767 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e9b60d6a-c27e-4d88-8df7-36585d72b57f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:11.613740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bda97be8-f483-4b01-98a2-f805413e907e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:11.614600 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bda97be8-f483-4b01-98a2-f805413e907e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:11.622849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bda97be8-f483-4b01-98a2-f805413e907e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:11.622849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bda97be8-f483-4b01-98a2-f805413e907e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:11.623247 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bda97be8-f483-4b01-98a2-f805413e907e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:11.624004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bda97be8-f483-4b01-98a2-f805413e907e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:11.624492 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bda97be8-f483-4b01-98a2-f805413e907e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:11.625171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bda97be8-f483-4b01-98a2-f805413e907e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:11.958946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=8 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:11.975956 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bda97be8-f483-4b01-98a2-f805413e907e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:11.984744 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e9b60d6a-c27e-4d88-8df7-36585d72b57f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:11.985150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bda97be8-f483-4b01-98a2-f805413e907e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1315 microversion: 2.1 time: 0.376534 Sep 08 13:54:11.998457 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e9b60d6a-c27e-4d88-8df7-36585d72b57f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.615116 Sep 08 13:54:11.999306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 52/106] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:11 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1315 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:11.999982 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 55/107] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:11 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 618 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:12.008771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-bdcf9535-dbd2-4772-9951-00227822c66f req-8241e736-3589-47ec-b94c-cd10959efdab service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "2707cec6-5b8c-41b5-8ae3-1b4c4de591af", "tag": "850abdba-93ff-4f41-832b-fbb4d1e81248"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:54:12.020445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-bdcf9535-dbd2-4772-9951-00227822c66f req-8241e736-3589-47ec-b94c-cd10959efdab service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:12.021263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-bdcf9535-dbd2-4772-9951-00227822c66f req-8241e736-3589-47ec-b94c-cd10959efdab service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:12.021916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-bdcf9535-dbd2-4772-9951-00227822c66f req-8241e736-3589-47ec-b94c-cd10959efdab service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:12.033690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-bdcf9535-dbd2-4772-9951-00227822c66f req-8241e736-3589-47ec-b94c-cd10959efdab service nova] Creating event network-changed:850abdba-93ff-4f41-832b-fbb4d1e81248 for instance 2707cec6-5b8c-41b5-8ae3-1b4c4de591af on hv-cinder-85646 Sep 08 13:54:12.041185 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-bdcf9535-dbd2-4772-9951-00227822c66f req-8241e736-3589-47ec-b94c-cd10959efdab service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040336 Sep 08 13:54:12.042513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 53/108] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:54:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:54:12.067088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-36908fc4-e0b5-4a7c-be19-00558872340f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:12.074345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-36908fc4-e0b5-4a7c-be19-00558872340f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:12.111039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-36908fc4-e0b5-4a7c-be19-00558872340f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:12.113075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-36908fc4-e0b5-4a7c-be19-00558872340f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:12.114476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-36908fc4-e0b5-4a7c-be19-00558872340f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:12.116486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-36908fc4-e0b5-4a7c-be19-00558872340f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:12.117420 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-36908fc4-e0b5-4a7c-be19-00558872340f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:12.118409 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-36908fc4-e0b5-4a7c-be19-00558872340f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:12.220669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-013119a1-e506-4f82-9c4c-936d7e36d561 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:12.221518 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-013119a1-e506-4f82-9c4c-936d7e36d561 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:12.229975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-013119a1-e506-4f82-9c4c-936d7e36d561 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:12.231844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-013119a1-e506-4f82-9c4c-936d7e36d561 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:12.232993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-013119a1-e506-4f82-9c4c-936d7e36d561 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:12.233931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-013119a1-e506-4f82-9c4c-936d7e36d561 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:12.234930 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-013119a1-e506-4f82-9c4c-936d7e36d561 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:12.235554 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-013119a1-e506-4f82-9c4c-936d7e36d561 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:12.346055 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-36908fc4-e0b5-4a7c-be19-00558872340f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:12.381149 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-36908fc4-e0b5-4a7c-be19-00558872340f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:12.382967 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-36908fc4-e0b5-4a7c-be19-00558872340f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:12.384465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-36908fc4-e0b5-4a7c-be19-00558872340f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1351 microversion: 2.20 time: 0.343568 Sep 08 13:54:12.385611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 56/109] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:12 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1351 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:54:12.577021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-013119a1-e506-4f82-9c4c-936d7e36d561 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:12.634383 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-013119a1-e506-4f82-9c4c-936d7e36d561 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1366 microversion: 2.1 time: 0.410770 Sep 08 13:54:12.648879 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 54/110] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:12 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1366 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:12.920141 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-8b2528cf-16eb-4701-bcde-30a5ff3130a5 req-ed996420-8647-4d3c-9916-be46b55bd1fe service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a", "tag": "a6d2c241-c0b3-4218-8498-e49114f1540f"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:54:12.952508 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-8b2528cf-16eb-4701-bcde-30a5ff3130a5 req-ed996420-8647-4d3c-9916-be46b55bd1fe service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:12.953884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-8b2528cf-16eb-4701-bcde-30a5ff3130a5 req-ed996420-8647-4d3c-9916-be46b55bd1fe service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:12.956067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-8b2528cf-16eb-4701-bcde-30a5ff3130a5 req-ed996420-8647-4d3c-9916-be46b55bd1fe service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:13.005883 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-8b2528cf-16eb-4701-bcde-30a5ff3130a5 req-ed996420-8647-4d3c-9916-be46b55bd1fe service nova] Creating event network-changed:a6d2c241-c0b3-4218-8498-e49114f1540f for instance dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a on hv-cinder-85646 Sep 08 13:54:13.014304 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0351fac2-8e6e-447c-bc28-c6a643e4bdad tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:13.015053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0351fac2-8e6e-447c-bc28-c6a643e4bdad tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:13.020466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-8b2528cf-16eb-4701-bcde-30a5ff3130a5 req-ed996420-8647-4d3c-9916-be46b55bd1fe service nova] Acquiring lock "compute-rpcapi-router" {{(pid=127602) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 08 13:54:13.020857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-8b2528cf-16eb-4701-bcde-30a5ff3130a5 req-ed996420-8647-4d3c-9916-be46b55bd1fe service nova] Acquired lock "compute-rpcapi-router" {{(pid=127602) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 08 13:54:13.021949 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc187469-d432-4b5d-92ad-2bc981c44aaa None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:13.023739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0351fac2-8e6e-447c-bc28-c6a643e4bdad tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:13.024261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0351fac2-8e6e-447c-bc28-c6a643e4bdad tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:13.024694 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0351fac2-8e6e-447c-bc28-c6a643e4bdad tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:13.025388 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc187469-d432-4b5d-92ad-2bc981c44aaa None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:13.025850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc187469-d432-4b5d-92ad-2bc981c44aaa None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:13.026790 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc187469-d432-4b5d-92ad-2bc981c44aaa None None] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:13.027240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc187469-d432-4b5d-92ad-2bc981c44aaa None None] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:13.028150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0351fac2-8e6e-447c-bc28-c6a643e4bdad tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:13.028743 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc187469-d432-4b5d-92ad-2bc981c44aaa None None] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:13.032475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0351fac2-8e6e-447c-bc28-c6a643e4bdad tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:13.032921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0351fac2-8e6e-447c-bc28-c6a643e4bdad tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:13.044826 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.compute.rpcapi [None req-fc187469-d432-4b5d-92ad-2bc981c44aaa None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 08 13:54:13.045912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc187469-d432-4b5d-92ad-2bc981c44aaa None None] Releasing lock "compute-rpcapi-router" {{(pid=127602) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 08 13:54:13.086850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fc187469-d432-4b5d-92ad-2bc981c44aaa None None] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.252052 Sep 08 13:54:13.088862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 57/111] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:54:12 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:54:13.102045 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ad97a086-cdef-4e45-9495-e6a26bedd3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:13.103778 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ad97a086-cdef-4e45-9495-e6a26bedd3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:13.117807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ad97a086-cdef-4e45-9495-e6a26bedd3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:13.118624 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ad97a086-cdef-4e45-9495-e6a26bedd3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:13.119217 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ad97a086-cdef-4e45-9495-e6a26bedd3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:13.120300 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ad97a086-cdef-4e45-9495-e6a26bedd3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:13.120935 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ad97a086-cdef-4e45-9495-e6a26bedd3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:13.121521 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ad97a086-cdef-4e45-9495-e6a26bedd3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:13.241998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0351fac2-8e6e-447c-bc28-c6a643e4bdad tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:13.252092 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0351fac2-8e6e-447c-bc28-c6a643e4bdad tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1315 microversion: 2.1 time: 0.242720 Sep 08 13:54:13.268663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 55/112] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:13 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1315 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:13.331396 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ad97a086-cdef-4e45-9495-e6a26bedd3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:13.334153 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ad97a086-cdef-4e45-9495-e6a26bedd3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.242766 Sep 08 13:54:13.334618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 58/113] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:13 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:13.409205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-adcd37fb-0d2e-4b66-9553-4feb337d0fa0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:13.409719 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-adcd37fb-0d2e-4b66-9553-4feb337d0fa0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:13.415888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-adcd37fb-0d2e-4b66-9553-4feb337d0fa0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:13.416369 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-adcd37fb-0d2e-4b66-9553-4feb337d0fa0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:13.416884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-adcd37fb-0d2e-4b66-9553-4feb337d0fa0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:13.418471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-adcd37fb-0d2e-4b66-9553-4feb337d0fa0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:13.419467 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-adcd37fb-0d2e-4b66-9553-4feb337d0fa0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:13.419889 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-adcd37fb-0d2e-4b66-9553-4feb337d0fa0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:13.592509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-adcd37fb-0d2e-4b66-9553-4feb337d0fa0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:13.602226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-adcd37fb-0d2e-4b66-9553-4feb337d0fa0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:13.604270 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-adcd37fb-0d2e-4b66-9553-4feb337d0fa0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:13.605440 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-adcd37fb-0d2e-4b66-9553-4feb337d0fa0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.200454 Sep 08 13:54:13.606732 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 56/114] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:13 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:13.673872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-82ca086f-9731-44c0-88e9-ea51c07c2417 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:13.674594 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-82ca086f-9731-44c0-88e9-ea51c07c2417 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:13.688321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-82ca086f-9731-44c0-88e9-ea51c07c2417 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:13.688321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-82ca086f-9731-44c0-88e9-ea51c07c2417 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:13.688321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-82ca086f-9731-44c0-88e9-ea51c07c2417 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:13.688321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-82ca086f-9731-44c0-88e9-ea51c07c2417 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:13.688619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-82ca086f-9731-44c0-88e9-ea51c07c2417 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:13.688619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-82ca086f-9731-44c0-88e9-ea51c07c2417 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:14.276612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fc001c70-c761-4b92-93ad-1621483f621f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:14.277491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fc001c70-c761-4b92-93ad-1621483f621f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:14.290396 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc001c70-c761-4b92-93ad-1621483f621f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:14.290887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc001c70-c761-4b92-93ad-1621483f621f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:14.291317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc001c70-c761-4b92-93ad-1621483f621f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:14.292018 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc001c70-c761-4b92-93ad-1621483f621f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:14.292466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc001c70-c761-4b92-93ad-1621483f621f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:14.292884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc001c70-c761-4b92-93ad-1621483f621f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:14.468223 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-82ca086f-9731-44c0-88e9-ea51c07c2417 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:14.480390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-82ca086f-9731-44c0-88e9-ea51c07c2417 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1366 microversion: 2.1 time: 0.812088 Sep 08 13:54:14.482094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 59/115] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:13 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1366 bytes in 814 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:14.486430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-08132202-14b9-43b9-b725-37788d63cc29 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:14.487234 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-08132202-14b9-43b9-b725-37788d63cc29 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:14.493751 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08132202-14b9-43b9-b725-37788d63cc29 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:14.494635 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08132202-14b9-43b9-b725-37788d63cc29 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:14.495248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08132202-14b9-43b9-b725-37788d63cc29 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:14.496120 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08132202-14b9-43b9-b725-37788d63cc29 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:14.497448 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08132202-14b9-43b9-b725-37788d63cc29 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:14.498058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08132202-14b9-43b9-b725-37788d63cc29 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:14.755247 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fc001c70-c761-4b92-93ad-1621483f621f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:14.765368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fc001c70-c761-4b92-93ad-1621483f621f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1315 microversion: 2.1 time: 0.492541 Sep 08 13:54:14.766486 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 57/116] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:14 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1315 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:14.773146 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7f359dfd-0e80-4f0f-9a04-0c26de80cc10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:14.774131 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7f359dfd-0e80-4f0f-9a04-0c26de80cc10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:14.782774 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f359dfd-0e80-4f0f-9a04-0c26de80cc10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:14.789271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f359dfd-0e80-4f0f-9a04-0c26de80cc10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:14.789271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f359dfd-0e80-4f0f-9a04-0c26de80cc10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:14.789271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f359dfd-0e80-4f0f-9a04-0c26de80cc10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:14.789271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f359dfd-0e80-4f0f-9a04-0c26de80cc10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:14.789649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f359dfd-0e80-4f0f-9a04-0c26de80cc10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:15.090180 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-08132202-14b9-43b9-b725-37788d63cc29 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:15.099476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-08132202-14b9-43b9-b725-37788d63cc29 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.617027 Sep 08 13:54:15.100705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 60/117] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:14 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 619 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:15.283754 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7f359dfd-0e80-4f0f-9a04-0c26de80cc10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:15.295453 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7f359dfd-0e80-4f0f-9a04-0c26de80cc10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:15.296906 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7f359dfd-0e80-4f0f-9a04-0c26de80cc10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:15.298540 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7f359dfd-0e80-4f0f-9a04-0c26de80cc10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.530868 Sep 08 13:54:15.299131 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 58/118] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:14 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:15.506753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8eef6ea4-0518-4a21-a3d5-23ed2e882459 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:15.506753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8eef6ea4-0518-4a21-a3d5-23ed2e882459 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:15.517667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8eef6ea4-0518-4a21-a3d5-23ed2e882459 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:15.517927 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8eef6ea4-0518-4a21-a3d5-23ed2e882459 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:15.518706 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8eef6ea4-0518-4a21-a3d5-23ed2e882459 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:15.520194 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8eef6ea4-0518-4a21-a3d5-23ed2e882459 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:15.520651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8eef6ea4-0518-4a21-a3d5-23ed2e882459 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:15.523112 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8eef6ea4-0518-4a21-a3d5-23ed2e882459 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:15.797512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3c0342da-c93c-44ce-8696-e834aba2e20c tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:15.799395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3c0342da-c93c-44ce-8696-e834aba2e20c tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:15.819085 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3c0342da-c93c-44ce-8696-e834aba2e20c tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:15.819851 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3c0342da-c93c-44ce-8696-e834aba2e20c tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:15.821628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3c0342da-c93c-44ce-8696-e834aba2e20c tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:15.822711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3c0342da-c93c-44ce-8696-e834aba2e20c tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:15.826761 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3c0342da-c93c-44ce-8696-e834aba2e20c tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:15.828719 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3c0342da-c93c-44ce-8696-e834aba2e20c tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:15.837835 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:15.863807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8eef6ea4-0518-4a21-a3d5-23ed2e882459 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:15.880745 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8eef6ea4-0518-4a21-a3d5-23ed2e882459 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1366 microversion: 2.1 time: 0.375011 Sep 08 13:54:15.881632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 61/119] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:15 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1366 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:15.997380 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3c0342da-c93c-44ce-8696-e834aba2e20c tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:16.008755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3c0342da-c93c-44ce-8696-e834aba2e20c tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1315 microversion: 2.1 time: 0.219349 Sep 08 13:54:16.009998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 59/120] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:15 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1315 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:16.137200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b3dbfdd1-0771-4953-941b-658ec3f6e10a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:16.139654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-0f6d33b9-84f6-4d6f-a692-7120a3c2553a req-ade1007c-c2fe-4a00-961b-d6a3b3426425 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "483c7eb7-3827-403f-84f3-c7c65dcbe574", "tag": "207fb2dc-4cc9-4491-b6db-03ae99699ac3"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:54:16.141515 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b3dbfdd1-0771-4953-941b-658ec3f6e10a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:16.185177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3dbfdd1-0771-4953-941b-658ec3f6e10a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:16.186088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3dbfdd1-0771-4953-941b-658ec3f6e10a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:16.187785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3dbfdd1-0771-4953-941b-658ec3f6e10a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:16.194915 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-0f6d33b9-84f6-4d6f-a692-7120a3c2553a req-ade1007c-c2fe-4a00-961b-d6a3b3426425 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:16.200235 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3dbfdd1-0771-4953-941b-658ec3f6e10a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:16.200235 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3dbfdd1-0771-4953-941b-658ec3f6e10a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:16.200235 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3dbfdd1-0771-4953-941b-658ec3f6e10a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:16.201263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-0f6d33b9-84f6-4d6f-a692-7120a3c2553a req-ade1007c-c2fe-4a00-961b-d6a3b3426425 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:16.203610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-0f6d33b9-84f6-4d6f-a692-7120a3c2553a req-ade1007c-c2fe-4a00-961b-d6a3b3426425 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:16.245084 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-0f6d33b9-84f6-4d6f-a692-7120a3c2553a req-ade1007c-c2fe-4a00-961b-d6a3b3426425 service nova] Creating event network-changed:207fb2dc-4cc9-4491-b6db-03ae99699ac3 for instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 on hv-cinder-85646 Sep 08 13:54:16.258988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-0f6d33b9-84f6-4d6f-a692-7120a3c2553a req-ade1007c-c2fe-4a00-961b-d6a3b3426425 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.129722 Sep 08 13:54:16.260928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 60/121] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:54:16 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:54:16.333125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-db6ea7ca-5e06-4cda-98bb-575446f5b7aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:16.335726 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-db6ea7ca-5e06-4cda-98bb-575446f5b7aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:16.349305 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db6ea7ca-5e06-4cda-98bb-575446f5b7aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:16.350206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db6ea7ca-5e06-4cda-98bb-575446f5b7aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:16.350845 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db6ea7ca-5e06-4cda-98bb-575446f5b7aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:16.351927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db6ea7ca-5e06-4cda-98bb-575446f5b7aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:16.352588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db6ea7ca-5e06-4cda-98bb-575446f5b7aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:16.353271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db6ea7ca-5e06-4cda-98bb-575446f5b7aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:16.563147 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-db6ea7ca-5e06-4cda-98bb-575446f5b7aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:16.568111 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-db6ea7ca-5e06-4cda-98bb-575446f5b7aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:16.570016 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-db6ea7ca-5e06-4cda-98bb-575446f5b7aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:16.571265 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-db6ea7ca-5e06-4cda-98bb-575446f5b7aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.248739 Sep 08 13:54:16.571901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 61/122] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:16 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:16.834458 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b3dbfdd1-0771-4953-941b-658ec3f6e10a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:16.847731 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b3dbfdd1-0771-4953-941b-658ec3f6e10a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.728976 Sep 08 13:54:16.848639 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 62/123] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:16 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 730 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:16.906640 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2f208ff6-58a8-4683-833f-d514a616ddeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:16.907562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2f208ff6-58a8-4683-833f-d514a616ddeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:16.913717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f208ff6-58a8-4683-833f-d514a616ddeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:16.914813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f208ff6-58a8-4683-833f-d514a616ddeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:16.915559 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f208ff6-58a8-4683-833f-d514a616ddeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:16.916782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f208ff6-58a8-4683-833f-d514a616ddeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:16.917638 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f208ff6-58a8-4683-833f-d514a616ddeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:16.918329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f208ff6-58a8-4683-833f-d514a616ddeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:17.082205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2f208ff6-58a8-4683-833f-d514a616ddeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:17.090729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2f208ff6-58a8-4683-833f-d514a616ddeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1366 microversion: 2.1 time: 0.188536 Sep 08 13:54:17.091439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 62/124] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:16 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1366 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:17.098520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5ffc3ab0-c1aa-4da2-bc96-8a6ee27d2c5f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:17.099123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5ffc3ab0-c1aa-4da2-bc96-8a6ee27d2c5f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:17.105178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5ffc3ab0-c1aa-4da2-bc96-8a6ee27d2c5f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:17.105793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5ffc3ab0-c1aa-4da2-bc96-8a6ee27d2c5f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:17.106770 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5ffc3ab0-c1aa-4da2-bc96-8a6ee27d2c5f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:17.107476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5ffc3ab0-c1aa-4da2-bc96-8a6ee27d2c5f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:17.107922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5ffc3ab0-c1aa-4da2-bc96-8a6ee27d2c5f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:17.108742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5ffc3ab0-c1aa-4da2-bc96-8a6ee27d2c5f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:17.180027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-b2628402-6c3d-46c0-ac9e-aff54ae3049d req-dbb70c2a-928c-442b-bf48-a4ad34fe1afb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "358e292a-9e96-4380-8ea5-3ed00372b4c3", "name": "network-vif-plugged", "status": "completed", "tag": "a0d989dd-29c8-414f-a850-15a4e3aea7a8"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:54:17.213962 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-b2628402-6c3d-46c0-ac9e-aff54ae3049d req-dbb70c2a-928c-442b-bf48-a4ad34fe1afb service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:17.218095 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-b2628402-6c3d-46c0-ac9e-aff54ae3049d req-dbb70c2a-928c-442b-bf48-a4ad34fe1afb service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:17.218774 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-b2628402-6c3d-46c0-ac9e-aff54ae3049d req-dbb70c2a-928c-442b-bf48-a4ad34fe1afb service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:17.251625 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-b2628402-6c3d-46c0-ac9e-aff54ae3049d req-dbb70c2a-928c-442b-bf48-a4ad34fe1afb service nova] Creating event network-vif-plugged:a0d989dd-29c8-414f-a850-15a4e3aea7a8 for instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 on hv-cinder-85646 Sep 08 13:54:17.269950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-b2628402-6c3d-46c0-ac9e-aff54ae3049d req-dbb70c2a-928c-442b-bf48-a4ad34fe1afb service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.234493 Sep 08 13:54:17.271531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 63/125] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:54:17 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:54:17.383572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5ffc3ab0-c1aa-4da2-bc96-8a6ee27d2c5f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:17.391917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5ffc3ab0-c1aa-4da2-bc96-8a6ee27d2c5f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1315 microversion: 2.1 time: 0.297002 Sep 08 13:54:17.398619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 63/126] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:17 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1315 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:17.603663 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3c74df42-2229-4426-a085-0a8bdc520c0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:17.604834 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3c74df42-2229-4426-a085-0a8bdc520c0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:17.616959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3c74df42-2229-4426-a085-0a8bdc520c0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:17.618970 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3c74df42-2229-4426-a085-0a8bdc520c0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:17.623560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3c74df42-2229-4426-a085-0a8bdc520c0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:17.626029 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3c74df42-2229-4426-a085-0a8bdc520c0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:17.626567 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3c74df42-2229-4426-a085-0a8bdc520c0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:17.628911 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3c74df42-2229-4426-a085-0a8bdc520c0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:17.673463 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:17.871065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-05141415-28b0-4f77-b8dd-4f5e41a73f24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:17.871675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-05141415-28b0-4f77-b8dd-4f5e41a73f24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:17.889799 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05141415-28b0-4f77-b8dd-4f5e41a73f24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:17.890332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05141415-28b0-4f77-b8dd-4f5e41a73f24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:17.890766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05141415-28b0-4f77-b8dd-4f5e41a73f24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:17.891481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05141415-28b0-4f77-b8dd-4f5e41a73f24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:17.891936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05141415-28b0-4f77-b8dd-4f5e41a73f24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:17.892356 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05141415-28b0-4f77-b8dd-4f5e41a73f24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:18.064694 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-05141415-28b0-4f77-b8dd-4f5e41a73f24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:18.073504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3c74df42-2229-4426-a085-0a8bdc520c0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:18.077920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-05141415-28b0-4f77-b8dd-4f5e41a73f24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.209860 Sep 08 13:54:18.082526 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3c74df42-2229-4426-a085-0a8bdc520c0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:18.083973 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3c74df42-2229-4426-a085-0a8bdc520c0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:18.089644 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3c74df42-2229-4426-a085-0a8bdc520c0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.490200 Sep 08 13:54:18.091069 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 64/128] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:17 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:54:18.094312 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 64/127] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:17 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:18.113831 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a2549631-afce-4ba1-bdaf-8b428186f231 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:18.114509 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a2549631-afce-4ba1-bdaf-8b428186f231 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:18.120848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a2549631-afce-4ba1-bdaf-8b428186f231 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:18.121342 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a2549631-afce-4ba1-bdaf-8b428186f231 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:18.122576 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a2549631-afce-4ba1-bdaf-8b428186f231 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:18.123581 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a2549631-afce-4ba1-bdaf-8b428186f231 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:18.124237 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a2549631-afce-4ba1-bdaf-8b428186f231 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:18.124870 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a2549631-afce-4ba1-bdaf-8b428186f231 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:18.298977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-3cf34b45-e4a3-4a0b-bbbb-4d92fff9c09d req-1fd05608-c808-447a-becc-fb930b0cf70e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c14f6817-e3ab-4a36-ba23-590498bf1b25", "tag": "4894f431-b6f6-46b3-95db-e9eb88227e33"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:54:18.308940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-3cf34b45-e4a3-4a0b-bbbb-4d92fff9c09d req-1fd05608-c808-447a-becc-fb930b0cf70e service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:18.309781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-3cf34b45-e4a3-4a0b-bbbb-4d92fff9c09d req-1fd05608-c808-447a-becc-fb930b0cf70e service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:18.310443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-3cf34b45-e4a3-4a0b-bbbb-4d92fff9c09d req-1fd05608-c808-447a-becc-fb930b0cf70e service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:18.322657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-3cf34b45-e4a3-4a0b-bbbb-4d92fff9c09d req-1fd05608-c808-447a-becc-fb930b0cf70e service nova] Creating event network-changed:4894f431-b6f6-46b3-95db-e9eb88227e33 for instance c14f6817-e3ab-4a36-ba23-590498bf1b25 on hv-cinder-85646 Sep 08 13:54:18.328277 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-3cf34b45-e4a3-4a0b-bbbb-4d92fff9c09d req-1fd05608-c808-447a-becc-fb930b0cf70e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032962 Sep 08 13:54:18.329574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 65/129] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:54:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:54:18.355050 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a2549631-afce-4ba1-bdaf-8b428186f231 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:18.361993 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a2549631-afce-4ba1-bdaf-8b428186f231 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1366 microversion: 2.1 time: 0.252370 Sep 08 13:54:18.362971 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 65/130] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:18 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1366 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:18.433817 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4b2b9e28-b87b-40be-ac00-1a9fef540e6a tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:18.434775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4b2b9e28-b87b-40be-ac00-1a9fef540e6a tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:18.441468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4b2b9e28-b87b-40be-ac00-1a9fef540e6a tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:18.442193 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4b2b9e28-b87b-40be-ac00-1a9fef540e6a tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:18.442774 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4b2b9e28-b87b-40be-ac00-1a9fef540e6a tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:18.443610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4b2b9e28-b87b-40be-ac00-1a9fef540e6a tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:18.444429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4b2b9e28-b87b-40be-ac00-1a9fef540e6a tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:18.445021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4b2b9e28-b87b-40be-ac00-1a9fef540e6a tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:18.670084 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4b2b9e28-b87b-40be-ac00-1a9fef540e6a tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:18.681962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4b2b9e28-b87b-40be-ac00-1a9fef540e6a tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1315 microversion: 2.1 time: 0.256415 Sep 08 13:54:18.687468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 66/131] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:18 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1315 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:19.100177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-28acd7b8-45ab-46b6-b3d1-2014af21bf13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:19.101001 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-28acd7b8-45ab-46b6-b3d1-2014af21bf13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:19.108871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-28acd7b8-45ab-46b6-b3d1-2014af21bf13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:19.115127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-50a4fa40-d00d-49be-8117-d31c444dc0fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:19.116991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-50a4fa40-d00d-49be-8117-d31c444dc0fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:19.118850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-28acd7b8-45ab-46b6-b3d1-2014af21bf13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:19.119354 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-28acd7b8-45ab-46b6-b3d1-2014af21bf13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:19.122932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50a4fa40-d00d-49be-8117-d31c444dc0fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:19.123437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50a4fa40-d00d-49be-8117-d31c444dc0fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:19.123851 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50a4fa40-d00d-49be-8117-d31c444dc0fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:19.124546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50a4fa40-d00d-49be-8117-d31c444dc0fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:19.125918 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-28acd7b8-45ab-46b6-b3d1-2014af21bf13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:19.126635 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-28acd7b8-45ab-46b6-b3d1-2014af21bf13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:19.127075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-28acd7b8-45ab-46b6-b3d1-2014af21bf13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:19.131576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50a4fa40-d00d-49be-8117-d31c444dc0fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:19.132091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50a4fa40-d00d-49be-8117-d31c444dc0fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:19.288972 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-28acd7b8-45ab-46b6-b3d1-2014af21bf13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:19.298721 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-28acd7b8-45ab-46b6-b3d1-2014af21bf13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.202396 Sep 08 13:54:19.298982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-50a4fa40-d00d-49be-8117-d31c444dc0fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:19.299393 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 66/132] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:19 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:19.305822 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-fde60676-10ae-443b-bf77-630cd0d7897b req-b35c0103-afdb-4e96-ae57-dfb7f2c609b7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2707cec6-5b8c-41b5-8ae3-1b4c4de591af", "name": "network-vif-plugged", "status": "completed", "tag": "850abdba-93ff-4f41-832b-fbb4d1e81248"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:54:19.307562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-50a4fa40-d00d-49be-8117-d31c444dc0fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:19.309735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-50a4fa40-d00d-49be-8117-d31c444dc0fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:19.310837 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-50a4fa40-d00d-49be-8117-d31c444dc0fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.199581 Sep 08 13:54:19.316985 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-fde60676-10ae-443b-bf77-630cd0d7897b req-b35c0103-afdb-4e96-ae57-dfb7f2c609b7 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:19.317544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-fde60676-10ae-443b-bf77-630cd0d7897b req-b35c0103-afdb-4e96-ae57-dfb7f2c609b7 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:19.317964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 67/133] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:19 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:19.318438 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-fde60676-10ae-443b-bf77-630cd0d7897b req-b35c0103-afdb-4e96-ae57-dfb7f2c609b7 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:19.340380 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-fde60676-10ae-443b-bf77-630cd0d7897b req-b35c0103-afdb-4e96-ae57-dfb7f2c609b7 service nova] Creating event network-vif-plugged:850abdba-93ff-4f41-832b-fbb4d1e81248 for instance 2707cec6-5b8c-41b5-8ae3-1b4c4de591af on hv-cinder-85646 Sep 08 13:54:19.346335 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-fde60676-10ae-443b-bf77-630cd0d7897b req-b35c0103-afdb-4e96-ae57-dfb7f2c609b7 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.045734 Sep 08 13:54:19.347390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 67/134] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:54:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:54:19.384702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b830ae7a-2586-4c77-a758-ebc72e3c383e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:19.385303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b830ae7a-2586-4c77-a758-ebc72e3c383e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:19.392263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b830ae7a-2586-4c77-a758-ebc72e3c383e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:19.392742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b830ae7a-2586-4c77-a758-ebc72e3c383e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:19.393169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b830ae7a-2586-4c77-a758-ebc72e3c383e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:19.393858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b830ae7a-2586-4c77-a758-ebc72e3c383e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:19.394324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b830ae7a-2586-4c77-a758-ebc72e3c383e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:19.394744 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b830ae7a-2586-4c77-a758-ebc72e3c383e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:19.557137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b830ae7a-2586-4c77-a758-ebc72e3c383e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:19.581645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b830ae7a-2586-4c77-a758-ebc72e3c383e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1366 microversion: 2.1 time: 0.201288 Sep 08 13:54:19.582264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 68/135] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:19 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1366 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:19.718824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5cc19236-54bd-4acc-86d0-083d667697a2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:19.721948 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5cc19236-54bd-4acc-86d0-083d667697a2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:19.732764 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5cc19236-54bd-4acc-86d0-083d667697a2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:19.733430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5cc19236-54bd-4acc-86d0-083d667697a2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:19.734075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5cc19236-54bd-4acc-86d0-083d667697a2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:19.734813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5cc19236-54bd-4acc-86d0-083d667697a2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:19.735431 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5cc19236-54bd-4acc-86d0-083d667697a2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:19.735863 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5cc19236-54bd-4acc-86d0-083d667697a2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:19.969003 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5cc19236-54bd-4acc-86d0-083d667697a2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:19.977601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5cc19236-54bd-4acc-86d0-083d667697a2 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1315 microversion: 2.1 time: 0.265047 Sep 08 13:54:19.978594 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 68/136] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:19 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1315 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:20.323358 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8e7097aa-5a66-40b0-85ec-b16af5330f22 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:20.323969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8e7097aa-5a66-40b0-85ec-b16af5330f22 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:20.330007 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8e7097aa-5a66-40b0-85ec-b16af5330f22 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:20.330764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8e7097aa-5a66-40b0-85ec-b16af5330f22 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:20.331952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8e7097aa-5a66-40b0-85ec-b16af5330f22 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:20.332806 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8e7097aa-5a66-40b0-85ec-b16af5330f22 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:20.333266 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8e7097aa-5a66-40b0-85ec-b16af5330f22 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:20.333693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8e7097aa-5a66-40b0-85ec-b16af5330f22 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:20.338723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3e34a1a9-fdb1-4d9d-86ab-7eb481b03c3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:20.340187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3e34a1a9-fdb1-4d9d-86ab-7eb481b03c3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:20.346442 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e34a1a9-fdb1-4d9d-86ab-7eb481b03c3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:20.347104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e34a1a9-fdb1-4d9d-86ab-7eb481b03c3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:20.347444 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e34a1a9-fdb1-4d9d-86ab-7eb481b03c3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:20.348132 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e34a1a9-fdb1-4d9d-86ab-7eb481b03c3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:20.348575 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e34a1a9-fdb1-4d9d-86ab-7eb481b03c3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:20.349793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e34a1a9-fdb1-4d9d-86ab-7eb481b03c3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:20.489199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8e7097aa-5a66-40b0-85ec-b16af5330f22 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:20.491547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3e34a1a9-fdb1-4d9d-86ab-7eb481b03c3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:20.498347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8e7097aa-5a66-40b0-85ec-b16af5330f22 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.178736 Sep 08 13:54:20.499675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 69/137] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:20 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:20.501325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3e34a1a9-fdb1-4d9d-86ab-7eb481b03c3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:20.503243 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3e34a1a9-fdb1-4d9d-86ab-7eb481b03c3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:20.504605 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3e34a1a9-fdb1-4d9d-86ab-7eb481b03c3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.171939 Sep 08 13:54:20.505823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 69/138] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:20 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:54:20.610938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7e5e41f6-c46c-40e1-8756-bbd1552cf5b8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:20.610938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7e5e41f6-c46c-40e1-8756-bbd1552cf5b8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:20.614437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7e5e41f6-c46c-40e1-8756-bbd1552cf5b8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:20.614919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7e5e41f6-c46c-40e1-8756-bbd1552cf5b8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:20.615354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7e5e41f6-c46c-40e1-8756-bbd1552cf5b8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:20.616104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7e5e41f6-c46c-40e1-8756-bbd1552cf5b8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:20.616563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7e5e41f6-c46c-40e1-8756-bbd1552cf5b8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:20.617024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7e5e41f6-c46c-40e1-8756-bbd1552cf5b8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:20.761089 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7e5e41f6-c46c-40e1-8756-bbd1552cf5b8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:20.778005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7e5e41f6-c46c-40e1-8756-bbd1552cf5b8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1366 microversion: 2.1 time: 0.176509 Sep 08 13:54:20.779266 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 70/139] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:20 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1366 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:21.001474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-87fc7791-d5ba-4087-a2ea-0bb459135d67 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:21.002177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-87fc7791-d5ba-4087-a2ea-0bb459135d67 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:21.008253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-87fc7791-d5ba-4087-a2ea-0bb459135d67 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:21.008735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-87fc7791-d5ba-4087-a2ea-0bb459135d67 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:21.009163 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-87fc7791-d5ba-4087-a2ea-0bb459135d67 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:21.009854 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-87fc7791-d5ba-4087-a2ea-0bb459135d67 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:21.010334 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-87fc7791-d5ba-4087-a2ea-0bb459135d67 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:21.010749 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-87fc7791-d5ba-4087-a2ea-0bb459135d67 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:21.163020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-87fc7791-d5ba-4087-a2ea-0bb459135d67 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:21.172939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-87fc7791-d5ba-4087-a2ea-0bb459135d67 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1315 microversion: 2.1 time: 0.174829 Sep 08 13:54:21.172939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 70/140] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:20 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1315 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:21.362858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-ddd379c3-d474-44cb-a842-393f19496606 req-94f9b9aa-e30e-48fe-b279-f0319d81fcf7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a", "name": "network-vif-plugged", "status": "completed", "tag": "a6d2c241-c0b3-4218-8498-e49114f1540f"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:54:21.372356 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-ddd379c3-d474-44cb-a842-393f19496606 req-94f9b9aa-e30e-48fe-b279-f0319d81fcf7 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:21.372896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-ddd379c3-d474-44cb-a842-393f19496606 req-94f9b9aa-e30e-48fe-b279-f0319d81fcf7 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:21.373376 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-ddd379c3-d474-44cb-a842-393f19496606 req-94f9b9aa-e30e-48fe-b279-f0319d81fcf7 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:21.386061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-ddd379c3-d474-44cb-a842-393f19496606 req-94f9b9aa-e30e-48fe-b279-f0319d81fcf7 service nova] Creating event network-vif-plugged:a6d2c241-c0b3-4218-8498-e49114f1540f for instance dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a on hv-cinder-85646 Sep 08 13:54:21.392130 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-ddd379c3-d474-44cb-a842-393f19496606 req-94f9b9aa-e30e-48fe-b279-f0319d81fcf7 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033113 Sep 08 13:54:21.392736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 71/141] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:54:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:54:21.526313 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-77d1e9e4-39a6-4cea-8170-dd687435c28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:21.526898 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-77d1e9e4-39a6-4cea-8170-dd687435c28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:21.531852 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-32754733-dfd0-48eb-b7b9-81c3dca16758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:21.541173 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-32754733-dfd0-48eb-b7b9-81c3dca16758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:21.541305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77d1e9e4-39a6-4cea-8170-dd687435c28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:21.541305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77d1e9e4-39a6-4cea-8170-dd687435c28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:21.541305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77d1e9e4-39a6-4cea-8170-dd687435c28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:21.541305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77d1e9e4-39a6-4cea-8170-dd687435c28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:21.541647 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77d1e9e4-39a6-4cea-8170-dd687435c28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:21.541647 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77d1e9e4-39a6-4cea-8170-dd687435c28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:21.541805 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32754733-dfd0-48eb-b7b9-81c3dca16758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:21.541805 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32754733-dfd0-48eb-b7b9-81c3dca16758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:21.541805 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32754733-dfd0-48eb-b7b9-81c3dca16758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:21.541805 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32754733-dfd0-48eb-b7b9-81c3dca16758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:21.542180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32754733-dfd0-48eb-b7b9-81c3dca16758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:21.542180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32754733-dfd0-48eb-b7b9-81c3dca16758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:21.695482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-32754733-dfd0-48eb-b7b9-81c3dca16758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:21.717873 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-32754733-dfd0-48eb-b7b9-81c3dca16758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:21.723807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-32754733-dfd0-48eb-b7b9-81c3dca16758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:21.726297 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-32754733-dfd0-48eb-b7b9-81c3dca16758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.199141 Sep 08 13:54:21.747392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 72/142] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:21 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:54:21.801536 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-aa1f9253-6595-46bd-bd6d-e8c8a868ac71 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:21.802160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-aa1f9253-6595-46bd-bd6d-e8c8a868ac71 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:21.808524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa1f9253-6595-46bd-bd6d-e8c8a868ac71 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:21.812348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa1f9253-6595-46bd-bd6d-e8c8a868ac71 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:21.812348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa1f9253-6595-46bd-bd6d-e8c8a868ac71 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:21.812348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa1f9253-6595-46bd-bd6d-e8c8a868ac71 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:21.812348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa1f9253-6595-46bd-bd6d-e8c8a868ac71 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:21.812676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa1f9253-6595-46bd-bd6d-e8c8a868ac71 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:21.965015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-77d1e9e4-39a6-4cea-8170-dd687435c28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:21.965506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-aa1f9253-6595-46bd-bd6d-e8c8a868ac71 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:21.976397 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-aa1f9253-6595-46bd-bd6d-e8c8a868ac71 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1366 microversion: 2.1 time: 0.178281 Sep 08 13:54:21.990655 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 73/143] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:21 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1366 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:21.990841 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-77d1e9e4-39a6-4cea-8170-dd687435c28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.459205 Sep 08 13:54:21.990841 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 71/144] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:21 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:22.207318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0b341ca8-8dc1-4f90-8583-b6bffc31de0f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:22.208361 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0b341ca8-8dc1-4f90-8583-b6bffc31de0f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:22.215526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b341ca8-8dc1-4f90-8583-b6bffc31de0f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:22.216003 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b341ca8-8dc1-4f90-8583-b6bffc31de0f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:22.216460 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b341ca8-8dc1-4f90-8583-b6bffc31de0f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:22.217303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b341ca8-8dc1-4f90-8583-b6bffc31de0f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:22.217694 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b341ca8-8dc1-4f90-8583-b6bffc31de0f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:22.218112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b341ca8-8dc1-4f90-8583-b6bffc31de0f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:22.414332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0b341ca8-8dc1-4f90-8583-b6bffc31de0f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:22.426380 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0b341ca8-8dc1-4f90-8583-b6bffc31de0f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1502 microversion: 2.1 time: 0.224398 Sep 08 13:54:22.426971 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 74/145] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:22 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1502 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:22.761097 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a72d0289-d21c-4c19-9cd0-025c0a164f1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:22.762038 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a72d0289-d21c-4c19-9cd0-025c0a164f1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:22.769798 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a72d0289-d21c-4c19-9cd0-025c0a164f1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:22.770319 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a72d0289-d21c-4c19-9cd0-025c0a164f1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:22.770886 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a72d0289-d21c-4c19-9cd0-025c0a164f1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:22.772068 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a72d0289-d21c-4c19-9cd0-025c0a164f1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:22.772522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a72d0289-d21c-4c19-9cd0-025c0a164f1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:22.772970 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a72d0289-d21c-4c19-9cd0-025c0a164f1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:22.923454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a72d0289-d21c-4c19-9cd0-025c0a164f1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:22.933808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a72d0289-d21c-4c19-9cd0-025c0a164f1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:22.935530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a72d0289-d21c-4c19-9cd0-025c0a164f1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:22.937030 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a72d0289-d21c-4c19-9cd0-025c0a164f1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.179592 Sep 08 13:54:22.938394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 72/146] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:22 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:22.999880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bd5dc774-0ed3-4658-9fc5-bed97b53415a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:23.000498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bd5dc774-0ed3-4658-9fc5-bed97b53415a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:23.005483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-866184e4-0d21-4dcd-a56f-da6639672a44 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:23.006418 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-866184e4-0d21-4dcd-a56f-da6639672a44 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:23.007104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd5dc774-0ed3-4658-9fc5-bed97b53415a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:23.007466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd5dc774-0ed3-4658-9fc5-bed97b53415a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:23.007884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd5dc774-0ed3-4658-9fc5-bed97b53415a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:23.008795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd5dc774-0ed3-4658-9fc5-bed97b53415a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:23.009439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd5dc774-0ed3-4658-9fc5-bed97b53415a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:23.009926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd5dc774-0ed3-4658-9fc5-bed97b53415a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:23.012465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-866184e4-0d21-4dcd-a56f-da6639672a44 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:23.012979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-866184e4-0d21-4dcd-a56f-da6639672a44 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:23.013414 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-866184e4-0d21-4dcd-a56f-da6639672a44 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:23.014392 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-866184e4-0d21-4dcd-a56f-da6639672a44 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:23.015048 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-866184e4-0d21-4dcd-a56f-da6639672a44 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:23.015875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-866184e4-0d21-4dcd-a56f-da6639672a44 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:23.043262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=2 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:23.199700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-866184e4-0d21-4dcd-a56f-da6639672a44 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:23.202861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bd5dc774-0ed3-4658-9fc5-bed97b53415a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:23.209169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bd5dc774-0ed3-4658-9fc5-bed97b53415a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1366 microversion: 2.1 time: 0.213897 Sep 08 13:54:23.210576 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-866184e4-0d21-4dcd-a56f-da6639672a44 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.210145 Sep 08 13:54:23.221539 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 75/147] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:22 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1366 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:23.224008 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 73/148] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:22 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:23.410595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-18040765-2ccd-4cdb-abbb-9efc1d3795dd req-4c5d28c8-b91f-41f5-a9d7-5e298dba2189 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "483c7eb7-3827-403f-84f3-c7c65dcbe574", "name": "network-vif-plugged", "status": "completed", "tag": "207fb2dc-4cc9-4491-b6db-03ae99699ac3"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:54:23.421413 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-18040765-2ccd-4cdb-abbb-9efc1d3795dd req-4c5d28c8-b91f-41f5-a9d7-5e298dba2189 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:23.422166 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-18040765-2ccd-4cdb-abbb-9efc1d3795dd req-4c5d28c8-b91f-41f5-a9d7-5e298dba2189 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:23.422888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-18040765-2ccd-4cdb-abbb-9efc1d3795dd req-4c5d28c8-b91f-41f5-a9d7-5e298dba2189 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:23.436474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-18040765-2ccd-4cdb-abbb-9efc1d3795dd req-4c5d28c8-b91f-41f5-a9d7-5e298dba2189 service nova] Creating event network-vif-plugged:207fb2dc-4cc9-4491-b6db-03ae99699ac3 for instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 on hv-cinder-85646 Sep 08 13:54:23.442222 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-18040765-2ccd-4cdb-abbb-9efc1d3795dd req-4c5d28c8-b91f-41f5-a9d7-5e298dba2189 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037532 Sep 08 13:54:23.442962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 76/149] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:54:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:54:23.961960 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ade396b9-1b84-497d-bf9f-6140b71de79c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:23.963188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ade396b9-1b84-497d-bf9f-6140b71de79c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:23.972922 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ade396b9-1b84-497d-bf9f-6140b71de79c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:23.973511 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ade396b9-1b84-497d-bf9f-6140b71de79c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:23.974072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ade396b9-1b84-497d-bf9f-6140b71de79c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:23.974627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ade396b9-1b84-497d-bf9f-6140b71de79c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:23.975131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ade396b9-1b84-497d-bf9f-6140b71de79c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:23.975628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ade396b9-1b84-497d-bf9f-6140b71de79c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:24.119678 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ade396b9-1b84-497d-bf9f-6140b71de79c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:24.128829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ade396b9-1b84-497d-bf9f-6140b71de79c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:24.130315 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ade396b9-1b84-497d-bf9f-6140b71de79c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:24.131428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ade396b9-1b84-497d-bf9f-6140b71de79c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.173921 Sep 08 13:54:24.132073 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 74/150] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:23 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:24.240107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ed2ec949-5488-41b7-b764-48df4643a8f5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:24.240919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ed2ec949-5488-41b7-b764-48df4643a8f5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:24.243150 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9cd99c71-d8c3-4aa8-9cd3-3473557e9bdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:24.244088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9cd99c71-d8c3-4aa8-9cd3-3473557e9bdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:24.248412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed2ec949-5488-41b7-b764-48df4643a8f5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:24.249264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed2ec949-5488-41b7-b764-48df4643a8f5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:24.250949 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9cd99c71-d8c3-4aa8-9cd3-3473557e9bdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:24.255681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9cd99c71-d8c3-4aa8-9cd3-3473557e9bdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:24.255681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9cd99c71-d8c3-4aa8-9cd3-3473557e9bdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:24.255681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9cd99c71-d8c3-4aa8-9cd3-3473557e9bdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:24.255681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9cd99c71-d8c3-4aa8-9cd3-3473557e9bdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:24.256135 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9cd99c71-d8c3-4aa8-9cd3-3473557e9bdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:24.256997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed2ec949-5488-41b7-b764-48df4643a8f5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:24.258080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed2ec949-5488-41b7-b764-48df4643a8f5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:24.258750 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed2ec949-5488-41b7-b764-48df4643a8f5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:24.259491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed2ec949-5488-41b7-b764-48df4643a8f5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:24.416525 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ed2ec949-5488-41b7-b764-48df4643a8f5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:24.426941 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ed2ec949-5488-41b7-b764-48df4643a8f5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.191025 Sep 08 13:54:24.428356 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 77/151] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:24 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:24.506024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-476ba934-0156-4762-b0dc-3045445f55e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "54441597-077c-4aab-8449-6d8124b60429"}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:54:24.509174 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-476ba934-0156-4762-b0dc-3045445f55e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:24.511583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9cd99c71-d8c3-4aa8-9cd3-3473557e9bdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:24.522087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-476ba934-0156-4762-b0dc-3045445f55e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:24.522087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-476ba934-0156-4762-b0dc-3045445f55e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:24.522087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-476ba934-0156-4762-b0dc-3045445f55e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:24.522087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-476ba934-0156-4762-b0dc-3045445f55e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:24.522658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-476ba934-0156-4762-b0dc-3045445f55e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:24.522658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-476ba934-0156-4762-b0dc-3045445f55e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:24.522796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9cd99c71-d8c3-4aa8-9cd3-3473557e9bdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1366 microversion: 2.1 time: 0.283317 Sep 08 13:54:24.522796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 75/152] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:24 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1366 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:24.599073 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:25.166525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-12202fd6-f4eb-42ec-bbd3-84409035d393 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:25.166525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-12202fd6-f4eb-42ec-bbd3-84409035d393 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:25.173289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-12202fd6-f4eb-42ec-bbd3-84409035d393 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:25.173994 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-12202fd6-f4eb-42ec-bbd3-84409035d393 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:25.174616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-12202fd6-f4eb-42ec-bbd3-84409035d393 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:25.175504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-12202fd6-f4eb-42ec-bbd3-84409035d393 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:25.176135 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-12202fd6-f4eb-42ec-bbd3-84409035d393 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:25.176594 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-12202fd6-f4eb-42ec-bbd3-84409035d393 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:25.335688 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-12202fd6-f4eb-42ec-bbd3-84409035d393 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:25.344256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-12202fd6-f4eb-42ec-bbd3-84409035d393 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:25.345688 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-12202fd6-f4eb-42ec-bbd3-84409035d393 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:25.346846 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-12202fd6-f4eb-42ec-bbd3-84409035d393 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.190353 Sep 08 13:54:25.354190 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 76/153] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:25 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:25.454499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4338548b-d7b4-4abb-9898-71c733d36294 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:25.455534 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4338548b-d7b4-4abb-9898-71c733d36294 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:25.461832 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4338548b-d7b4-4abb-9898-71c733d36294 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:25.462356 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4338548b-d7b4-4abb-9898-71c733d36294 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:25.462874 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4338548b-d7b4-4abb-9898-71c733d36294 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:25.467778 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4338548b-d7b4-4abb-9898-71c733d36294 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:25.467778 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4338548b-d7b4-4abb-9898-71c733d36294 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:25.467778 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4338548b-d7b4-4abb-9898-71c733d36294 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:25.608619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4338548b-d7b4-4abb-9898-71c733d36294 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:25.616229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4338548b-d7b4-4abb-9898-71c733d36294 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1321 microversion: 2.1 time: 0.166033 Sep 08 13:54:25.616794 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 77/154] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:25 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1321 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:25.621491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-926e2c96-6000-4113-aa3c-814719dc1d09 req-4cac27dd-993c-4983-af9d-2f6f160155e9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c14f6817-e3ab-4a36-ba23-590498bf1b25", "name": "network-vif-plugged", "status": "completed", "tag": "4894f431-b6f6-46b3-95db-e9eb88227e33"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:54:25.631312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-926e2c96-6000-4113-aa3c-814719dc1d09 req-4cac27dd-993c-4983-af9d-2f6f160155e9 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:25.631788 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-926e2c96-6000-4113-aa3c-814719dc1d09 req-4cac27dd-993c-4983-af9d-2f6f160155e9 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:25.632217 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-926e2c96-6000-4113-aa3c-814719dc1d09 req-4cac27dd-993c-4983-af9d-2f6f160155e9 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:25.650078 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-926e2c96-6000-4113-aa3c-814719dc1d09 req-4cac27dd-993c-4983-af9d-2f6f160155e9 service nova] Creating event network-vif-plugged:4894f431-b6f6-46b3-95db-e9eb88227e33 for instance c14f6817-e3ab-4a36-ba23-590498bf1b25 on hv-cinder-85646 Sep 08 13:54:25.655218 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-926e2c96-6000-4113-aa3c-814719dc1d09 req-4cac27dd-993c-4983-af9d-2f6f160155e9 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037234 Sep 08 13:54:25.656065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 78/155] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:54:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:54:25.666063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-88729a09-2348-4c62-900a-a9411bcc304b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:25.666063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-88729a09-2348-4c62-900a-a9411bcc304b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:25.668809 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88729a09-2348-4c62-900a-a9411bcc304b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:25.669523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88729a09-2348-4c62-900a-a9411bcc304b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:25.669921 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88729a09-2348-4c62-900a-a9411bcc304b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:25.670638 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88729a09-2348-4c62-900a-a9411bcc304b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:25.671089 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88729a09-2348-4c62-900a-a9411bcc304b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:25.671501 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88729a09-2348-4c62-900a-a9411bcc304b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:25.898552 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-88729a09-2348-4c62-900a-a9411bcc304b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:25.908328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-88729a09-2348-4c62-900a-a9411bcc304b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1366 microversion: 2.1 time: 0.251066 Sep 08 13:54:25.908794 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 79/156] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:25 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1366 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:26.375086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6d30d342-2307-4a8f-8673-9a5edc488b47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:26.375733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6d30d342-2307-4a8f-8673-9a5edc488b47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:26.381541 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d30d342-2307-4a8f-8673-9a5edc488b47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:26.382027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d30d342-2307-4a8f-8673-9a5edc488b47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:26.382498 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d30d342-2307-4a8f-8673-9a5edc488b47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:26.383190 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d30d342-2307-4a8f-8673-9a5edc488b47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:26.383647 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d30d342-2307-4a8f-8673-9a5edc488b47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:26.384070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d30d342-2307-4a8f-8673-9a5edc488b47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:26.522204 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6d30d342-2307-4a8f-8673-9a5edc488b47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:26.531319 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6d30d342-2307-4a8f-8673-9a5edc488b47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:26.533270 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6d30d342-2307-4a8f-8673-9a5edc488b47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:26.535096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6d30d342-2307-4a8f-8673-9a5edc488b47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.163640 Sep 08 13:54:26.536396 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 80/157] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:26 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:54:26.639165 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-19775ea8-2262-41f2-8ab9-b3fea6548218 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:26.639767 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-19775ea8-2262-41f2-8ab9-b3fea6548218 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:26.647823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19775ea8-2262-41f2-8ab9-b3fea6548218 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:26.648302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19775ea8-2262-41f2-8ab9-b3fea6548218 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:26.648731 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19775ea8-2262-41f2-8ab9-b3fea6548218 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:26.649416 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19775ea8-2262-41f2-8ab9-b3fea6548218 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:26.649865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19775ea8-2262-41f2-8ab9-b3fea6548218 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:26.650309 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19775ea8-2262-41f2-8ab9-b3fea6548218 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:26.819461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-19775ea8-2262-41f2-8ab9-b3fea6548218 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:26.828042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-19775ea8-2262-41f2-8ab9-b3fea6548218 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1514 microversion: 2.1 time: 0.192726 Sep 08 13:54:26.829589 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 81/158] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:26 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1514 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:26.854090 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5c240a29-1867-4804-9a50-c1546bec2c48 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:54:26.857257 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5c240a29-1867-4804-9a50-c1546bec2c48 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:26.863460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5c240a29-1867-4804-9a50-c1546bec2c48 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:26.863948 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5c240a29-1867-4804-9a50-c1546bec2c48 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:26.864378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5c240a29-1867-4804-9a50-c1546bec2c48 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:26.865074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5c240a29-1867-4804-9a50-c1546bec2c48 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:26.865525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5c240a29-1867-4804-9a50-c1546bec2c48 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:26.865944 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5c240a29-1867-4804-9a50-c1546bec2c48 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:26.923965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:26.930825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-5c240a29-1867-4804-9a50-c1546bec2c48 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lazy-loading 'flavor' on Instance uuid dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 13:54:27.333706 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5c240a29-1867-4804-9a50-c1546bec2c48 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "POST /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a/action" status: 200 len: 32 microversion: 2.1 time: 0.482921 Sep 08 13:54:27.338179 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 82/159] 10.222.0.25 () {60 vars in 1278 bytes} [Thu Sep 8 13:54:26 2022] POST /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a/action => generated 32 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 08 13:54:27.351879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-74afdfb5-70f1-4031-87f4-77ada0e54e44 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:27.352589 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-74afdfb5-70f1-4031-87f4-77ada0e54e44 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:27.365630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74afdfb5-70f1-4031-87f4-77ada0e54e44 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:27.366321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74afdfb5-70f1-4031-87f4-77ada0e54e44 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:27.367703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74afdfb5-70f1-4031-87f4-77ada0e54e44 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:27.369524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74afdfb5-70f1-4031-87f4-77ada0e54e44 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:27.371336 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74afdfb5-70f1-4031-87f4-77ada0e54e44 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:27.372975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74afdfb5-70f1-4031-87f4-77ada0e54e44 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:27.541858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-74afdfb5-70f1-4031-87f4-77ada0e54e44 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:27.553803 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-74afdfb5-70f1-4031-87f4-77ada0e54e44 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1366 microversion: 2.1 time: 0.214396 Sep 08 13:54:27.554679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 83/160] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:27 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1366 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:27.560764 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-dcda27eb-a7ef-4a4c-a562-e55a146d956d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:27.566257 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-dcda27eb-a7ef-4a4c-a562-e55a146d956d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:27.569884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dcda27eb-a7ef-4a4c-a562-e55a146d956d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:27.570560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dcda27eb-a7ef-4a4c-a562-e55a146d956d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:27.571288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dcda27eb-a7ef-4a4c-a562-e55a146d956d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:27.571996 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dcda27eb-a7ef-4a4c-a562-e55a146d956d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:27.572446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dcda27eb-a7ef-4a4c-a562-e55a146d956d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:27.573032 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dcda27eb-a7ef-4a4c-a562-e55a146d956d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:27.726619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-dcda27eb-a7ef-4a4c-a562-e55a146d956d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:27.735443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-dcda27eb-a7ef-4a4c-a562-e55a146d956d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.178953 Sep 08 13:54:27.738299 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 84/161] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:27 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:27.743186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-47dbdf5b-0eac-4153-8566-417a3f64a5fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:27.743791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-47dbdf5b-0eac-4153-8566-417a3f64a5fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:27.751866 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-47dbdf5b-0eac-4153-8566-417a3f64a5fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:27.752349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-47dbdf5b-0eac-4153-8566-417a3f64a5fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:27.752781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-47dbdf5b-0eac-4153-8566-417a3f64a5fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:27.753471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-47dbdf5b-0eac-4153-8566-417a3f64a5fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:27.753925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-47dbdf5b-0eac-4153-8566-417a3f64a5fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:27.754363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-47dbdf5b-0eac-4153-8566-417a3f64a5fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:27.901783 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-47dbdf5b-0eac-4153-8566-417a3f64a5fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:27.910715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-47dbdf5b-0eac-4153-8566-417a3f64a5fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:27.912615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-47dbdf5b-0eac-4153-8566-417a3f64a5fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:27.913760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-47dbdf5b-0eac-4153-8566-417a3f64a5fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.174315 Sep 08 13:54:27.914340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 85/162] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:27 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:27.976687 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-476ba934-0156-4762-b0dc-3045445f55e8 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "POST /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 3.474625 Sep 08 13:54:27.982572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 78/163] 10.222.0.25 () {60 vars in 1323 bytes} [Thu Sep 8 13:54:24 2022] POST /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3/os-volume_attachments => generated 194 bytes in 3481 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:54:28.577484 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e4e9b03b-800d-43f5-a6d4-17d1bcf27af1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:28.578173 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e4e9b03b-800d-43f5-a6d4-17d1bcf27af1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:28.584051 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4e9b03b-800d-43f5-a6d4-17d1bcf27af1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:28.584530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4e9b03b-800d-43f5-a6d4-17d1bcf27af1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:28.584959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4e9b03b-800d-43f5-a6d4-17d1bcf27af1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:28.585639 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4e9b03b-800d-43f5-a6d4-17d1bcf27af1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:28.586085 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4e9b03b-800d-43f5-a6d4-17d1bcf27af1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:28.586528 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4e9b03b-800d-43f5-a6d4-17d1bcf27af1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:28.738368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e4e9b03b-800d-43f5-a6d4-17d1bcf27af1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:28.746756 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e4e9b03b-800d-43f5-a6d4-17d1bcf27af1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1366 microversion: 2.1 time: 0.173678 Sep 08 13:54:28.748379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 86/164] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:28 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1366 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:28.771012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bce60df7-e106-4d5e-9c31-3f9c8085acce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:28.771925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bce60df7-e106-4d5e-9c31-3f9c8085acce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:28.778277 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bce60df7-e106-4d5e-9c31-3f9c8085acce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:28.778799 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bce60df7-e106-4d5e-9c31-3f9c8085acce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:28.779235 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bce60df7-e106-4d5e-9c31-3f9c8085acce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:28.779926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bce60df7-e106-4d5e-9c31-3f9c8085acce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:28.780381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bce60df7-e106-4d5e-9c31-3f9c8085acce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:28.781919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bce60df7-e106-4d5e-9c31-3f9c8085acce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:28.924010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bce60df7-e106-4d5e-9c31-3f9c8085acce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:28.932824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bce60df7-e106-4d5e-9c31-3f9c8085acce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.166696 Sep 08 13:54:28.940370 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 79/165] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:28 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:28.945221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7173c18b-3e75-40a3-8b1b-4a11adc0a12c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:28.945833 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7173c18b-3e75-40a3-8b1b-4a11adc0a12c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:28.952695 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7173c18b-3e75-40a3-8b1b-4a11adc0a12c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:28.953208 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7173c18b-3e75-40a3-8b1b-4a11adc0a12c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:28.953641 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7173c18b-3e75-40a3-8b1b-4a11adc0a12c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:28.954361 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7173c18b-3e75-40a3-8b1b-4a11adc0a12c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:28.954810 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7173c18b-3e75-40a3-8b1b-4a11adc0a12c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:28.955358 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7173c18b-3e75-40a3-8b1b-4a11adc0a12c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:29.089532 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7173c18b-3e75-40a3-8b1b-4a11adc0a12c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:29.099442 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7173c18b-3e75-40a3-8b1b-4a11adc0a12c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:29.102102 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7173c18b-3e75-40a3-8b1b-4a11adc0a12c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:29.103568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7173c18b-3e75-40a3-8b1b-4a11adc0a12c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.162105 Sep 08 13:54:29.109733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 87/166] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:28 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:29.775351 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-70bf7aca-528d-486f-a858-97570725262d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:29.776325 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-70bf7aca-528d-486f-a858-97570725262d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:29.783149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70bf7aca-528d-486f-a858-97570725262d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:29.783854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70bf7aca-528d-486f-a858-97570725262d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:29.784287 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70bf7aca-528d-486f-a858-97570725262d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:29.784983 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70bf7aca-528d-486f-a858-97570725262d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:29.785425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70bf7aca-528d-486f-a858-97570725262d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:29.785847 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70bf7aca-528d-486f-a858-97570725262d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:29.931592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-70bf7aca-528d-486f-a858-97570725262d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:29.943960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-70bf7aca-528d-486f-a858-97570725262d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1366 microversion: 2.1 time: 0.171942 Sep 08 13:54:29.945031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 80/167] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:29 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1366 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:29.956383 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7df706f3-aa72-4b99-9296-d2c9f680236b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:29.957795 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7df706f3-aa72-4b99-9296-d2c9f680236b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:29.964571 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7df706f3-aa72-4b99-9296-d2c9f680236b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:29.965134 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7df706f3-aa72-4b99-9296-d2c9f680236b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:29.965652 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7df706f3-aa72-4b99-9296-d2c9f680236b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:29.967233 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7df706f3-aa72-4b99-9296-d2c9f680236b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:29.967800 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7df706f3-aa72-4b99-9296-d2c9f680236b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:29.968312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7df706f3-aa72-4b99-9296-d2c9f680236b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:30.121157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7df706f3-aa72-4b99-9296-d2c9f680236b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:30.126527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7df706f3-aa72-4b99-9296-d2c9f680236b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.174201 Sep 08 13:54:30.130580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 88/168] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:29 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:30.141833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-65137123-8e01-40d6-81f4-f7fd4dcfc7ab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:30.142593 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-65137123-8e01-40d6-81f4-f7fd4dcfc7ab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:30.149012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-65137123-8e01-40d6-81f4-f7fd4dcfc7ab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:30.149488 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-65137123-8e01-40d6-81f4-f7fd4dcfc7ab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:30.149912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-65137123-8e01-40d6-81f4-f7fd4dcfc7ab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:30.150629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-65137123-8e01-40d6-81f4-f7fd4dcfc7ab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:30.151073 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-65137123-8e01-40d6-81f4-f7fd4dcfc7ab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:30.151485 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-65137123-8e01-40d6-81f4-f7fd4dcfc7ab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:30.326223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-65137123-8e01-40d6-81f4-f7fd4dcfc7ab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:30.338813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-65137123-8e01-40d6-81f4-f7fd4dcfc7ab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:30.340317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-65137123-8e01-40d6-81f4-f7fd4dcfc7ab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:30.341440 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-65137123-8e01-40d6-81f4-f7fd4dcfc7ab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.204804 Sep 08 13:54:30.342190 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 81/169] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:30 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:30.975168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-54579f7a-6c96-4265-aea2-7a3c47359a49 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:30.975772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-54579f7a-6c96-4265-aea2-7a3c47359a49 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:30.982395 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-54579f7a-6c96-4265-aea2-7a3c47359a49 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:30.982929 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-54579f7a-6c96-4265-aea2-7a3c47359a49 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:30.983368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-54579f7a-6c96-4265-aea2-7a3c47359a49 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:30.984140 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-54579f7a-6c96-4265-aea2-7a3c47359a49 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:30.984597 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-54579f7a-6c96-4265-aea2-7a3c47359a49 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:30.985012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-54579f7a-6c96-4265-aea2-7a3c47359a49 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:31.127193 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-54579f7a-6c96-4265-aea2-7a3c47359a49 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:31.135182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-54579f7a-6c96-4265-aea2-7a3c47359a49 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1366 microversion: 2.1 time: 0.163791 Sep 08 13:54:31.136009 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 89/170] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:30 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1366 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:31.151030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7b7aeceb-9b6c-4c3b-a847-71202541bb9f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:31.152288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7b7aeceb-9b6c-4c3b-a847-71202541bb9f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:31.159409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b7aeceb-9b6c-4c3b-a847-71202541bb9f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:31.159909 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b7aeceb-9b6c-4c3b-a847-71202541bb9f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:31.160346 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b7aeceb-9b6c-4c3b-a847-71202541bb9f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:31.161051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b7aeceb-9b6c-4c3b-a847-71202541bb9f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:31.161544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b7aeceb-9b6c-4c3b-a847-71202541bb9f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:31.162115 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b7aeceb-9b6c-4c3b-a847-71202541bb9f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:31.328939 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7b7aeceb-9b6c-4c3b-a847-71202541bb9f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:31.337284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7b7aeceb-9b6c-4c3b-a847-71202541bb9f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.190059 Sep 08 13:54:31.337866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 82/171] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:31 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:31.370130 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-18f48c78-d0a2-4810-9730-bc2cc129aa0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:31.370688 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-18f48c78-d0a2-4810-9730-bc2cc129aa0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:31.377099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18f48c78-d0a2-4810-9730-bc2cc129aa0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:31.377628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18f48c78-d0a2-4810-9730-bc2cc129aa0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:31.378265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18f48c78-d0a2-4810-9730-bc2cc129aa0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:31.379161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18f48c78-d0a2-4810-9730-bc2cc129aa0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:31.379617 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18f48c78-d0a2-4810-9730-bc2cc129aa0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:31.380055 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18f48c78-d0a2-4810-9730-bc2cc129aa0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:31.531383 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-18f48c78-d0a2-4810-9730-bc2cc129aa0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:31.540174 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-18f48c78-d0a2-4810-9730-bc2cc129aa0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:31.542064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-18f48c78-d0a2-4810-9730-bc2cc129aa0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:31.543901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-18f48c78-d0a2-4810-9730-bc2cc129aa0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.177099 Sep 08 13:54:31.543901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 90/172] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:31 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:32.162519 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2d4409ea-ae6f-4074-9224-eba88d34a096 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:32.163762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2d4409ea-ae6f-4074-9224-eba88d34a096 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:32.173105 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d4409ea-ae6f-4074-9224-eba88d34a096 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:32.174670 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d4409ea-ae6f-4074-9224-eba88d34a096 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:32.175270 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d4409ea-ae6f-4074-9224-eba88d34a096 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:32.176757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d4409ea-ae6f-4074-9224-eba88d34a096 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:32.177476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d4409ea-ae6f-4074-9224-eba88d34a096 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:32.178087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d4409ea-ae6f-4074-9224-eba88d34a096 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:32.337968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2d4409ea-ae6f-4074-9224-eba88d34a096 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:32.347900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2d4409ea-ae6f-4074-9224-eba88d34a096 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1366 microversion: 2.1 time: 0.192322 Sep 08 13:54:32.348489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 83/173] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:32 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1366 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:32.362614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f86fea63-628c-4127-bd43-3e7077e5a427 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:32.363784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f86fea63-628c-4127-bd43-3e7077e5a427 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:32.375719 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:32.378299 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f86fea63-628c-4127-bd43-3e7077e5a427 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:32.378963 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f86fea63-628c-4127-bd43-3e7077e5a427 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:32.379591 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f86fea63-628c-4127-bd43-3e7077e5a427 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:32.380832 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f86fea63-628c-4127-bd43-3e7077e5a427 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:32.381472 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f86fea63-628c-4127-bd43-3e7077e5a427 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:32.381976 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f86fea63-628c-4127-bd43-3e7077e5a427 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:32.532647 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f86fea63-628c-4127-bd43-3e7077e5a427 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:32.541272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f86fea63-628c-4127-bd43-3e7077e5a427 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.182387 Sep 08 13:54:32.550379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 91/174] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:32 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:32.567816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-56917fac-c494-4f72-ac08-facbdb39c857 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:32.568420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-56917fac-c494-4f72-ac08-facbdb39c857 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:32.574957 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56917fac-c494-4f72-ac08-facbdb39c857 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:32.575607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56917fac-c494-4f72-ac08-facbdb39c857 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:32.576224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56917fac-c494-4f72-ac08-facbdb39c857 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:32.577178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56917fac-c494-4f72-ac08-facbdb39c857 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:32.577636 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56917fac-c494-4f72-ac08-facbdb39c857 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:32.578412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56917fac-c494-4f72-ac08-facbdb39c857 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:32.718702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-56917fac-c494-4f72-ac08-facbdb39c857 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:32.728890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-56917fac-c494-4f72-ac08-facbdb39c857 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:32.734689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-56917fac-c494-4f72-ac08-facbdb39c857 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:32.734689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-56917fac-c494-4f72-ac08-facbdb39c857 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.168935 Sep 08 13:54:32.734689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 84/175] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:32 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:33.371569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d836c9f4-ae41-4649-a711-1709ce52094e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:33.372920 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d836c9f4-ae41-4649-a711-1709ce52094e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:33.379167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d836c9f4-ae41-4649-a711-1709ce52094e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:33.379658 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d836c9f4-ae41-4649-a711-1709ce52094e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:33.380093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d836c9f4-ae41-4649-a711-1709ce52094e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:33.380783 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d836c9f4-ae41-4649-a711-1709ce52094e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:33.381240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d836c9f4-ae41-4649-a711-1709ce52094e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:33.381665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d836c9f4-ae41-4649-a711-1709ce52094e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:33.525023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d836c9f4-ae41-4649-a711-1709ce52094e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:33.533110 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d836c9f4-ae41-4649-a711-1709ce52094e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1555 microversion: 2.1 time: 0.165475 Sep 08 13:54:33.533687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 92/176] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:33 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1555 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:33.556213 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-12b95c10-0c5d-4085-8bc3-61ca309e3185 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:33.556744 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-12b95c10-0c5d-4085-8bc3-61ca309e3185 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:33.563918 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-12b95c10-0c5d-4085-8bc3-61ca309e3185 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:33.564398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-12b95c10-0c5d-4085-8bc3-61ca309e3185 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:33.564820 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-12b95c10-0c5d-4085-8bc3-61ca309e3185 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:33.565186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-290fb1e4-b1d8-429b-9c1e-7e472527710e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:33.577491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-12b95c10-0c5d-4085-8bc3-61ca309e3185 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:33.577622 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-290fb1e4-b1d8-429b-9c1e-7e472527710e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:33.577714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-12b95c10-0c5d-4085-8bc3-61ca309e3185 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:33.577714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-12b95c10-0c5d-4085-8bc3-61ca309e3185 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:33.577857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-290fb1e4-b1d8-429b-9c1e-7e472527710e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:33.577857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-290fb1e4-b1d8-429b-9c1e-7e472527710e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:33.577857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-290fb1e4-b1d8-429b-9c1e-7e472527710e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:33.577857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-290fb1e4-b1d8-429b-9c1e-7e472527710e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:33.578188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-290fb1e4-b1d8-429b-9c1e-7e472527710e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:33.578188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-290fb1e4-b1d8-429b-9c1e-7e472527710e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:33.653215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-12b95c10-0c5d-4085-8bc3-61ca309e3185 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.100846 Sep 08 13:54:33.653831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 85/177] 10.222.0.25 () {58 vars in 1275 bytes} [Thu Sep 8 13:54:33 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-interface => generated 284 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:54:33.720218 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-290fb1e4-b1d8-429b-9c1e-7e472527710e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:33.731306 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-290fb1e4-b1d8-429b-9c1e-7e472527710e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.169857 Sep 08 13:54:33.732746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 93/178] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:33 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:33.757503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-98e57e02-48ca-4210-94e1-e3fbf6724f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:33.758855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-98e57e02-48ca-4210-94e1-e3fbf6724f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:33.766965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-98e57e02-48ca-4210-94e1-e3fbf6724f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:33.767918 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-98e57e02-48ca-4210-94e1-e3fbf6724f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:33.768651 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-98e57e02-48ca-4210-94e1-e3fbf6724f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:33.770978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-98e57e02-48ca-4210-94e1-e3fbf6724f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:33.771686 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-98e57e02-48ca-4210-94e1-e3fbf6724f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:33.772596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-98e57e02-48ca-4210-94e1-e3fbf6724f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:34.071334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-98e57e02-48ca-4210-94e1-e3fbf6724f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:34.080131 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-98e57e02-48ca-4210-94e1-e3fbf6724f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:34.081628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-98e57e02-48ca-4210-94e1-e3fbf6724f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:34.082944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-98e57e02-48ca-4210-94e1-e3fbf6724f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.329908 Sep 08 13:54:34.098533 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 86/179] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:33 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:34.755816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f21834f1-b35d-44f6-9afe-b54bacddacbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:34.756338 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f21834f1-b35d-44f6-9afe-b54bacddacbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:34.762370 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f21834f1-b35d-44f6-9afe-b54bacddacbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:34.762909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f21834f1-b35d-44f6-9afe-b54bacddacbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:34.763917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f21834f1-b35d-44f6-9afe-b54bacddacbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:34.764851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f21834f1-b35d-44f6-9afe-b54bacddacbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:34.765311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f21834f1-b35d-44f6-9afe-b54bacddacbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:34.765732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f21834f1-b35d-44f6-9afe-b54bacddacbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:34.909883 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f21834f1-b35d-44f6-9afe-b54bacddacbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:34.921852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f21834f1-b35d-44f6-9afe-b54bacddacbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.169102 Sep 08 13:54:34.924369 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 94/180] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:34 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:35.109702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-26942ad3-4e10-464e-a423-64bb41ecc280 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:35.110925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-26942ad3-4e10-464e-a423-64bb41ecc280 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:35.119253 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26942ad3-4e10-464e-a423-64bb41ecc280 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:35.119968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26942ad3-4e10-464e-a423-64bb41ecc280 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:35.120410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26942ad3-4e10-464e-a423-64bb41ecc280 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:35.121113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26942ad3-4e10-464e-a423-64bb41ecc280 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:35.121951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26942ad3-4e10-464e-a423-64bb41ecc280 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:35.122365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26942ad3-4e10-464e-a423-64bb41ecc280 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:35.364567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-26942ad3-4e10-464e-a423-64bb41ecc280 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:35.377765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-26942ad3-4e10-464e-a423-64bb41ecc280 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:35.378167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-8ba6094e-2f14-4eac-bead-e623f567b1ea req-cf75b127-0927-46d5-8db9-d5a8c9785a7b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "483c7eb7-3827-403f-84f3-c7c65dcbe574", "tag": "207fb2dc-4cc9-4491-b6db-03ae99699ac3"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:54:35.379857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-26942ad3-4e10-464e-a423-64bb41ecc280 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:35.381166 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-26942ad3-4e10-464e-a423-64bb41ecc280 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.276151 Sep 08 13:54:35.391489 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-8ba6094e-2f14-4eac-bead-e623f567b1ea req-cf75b127-0927-46d5-8db9-d5a8c9785a7b service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:35.392472 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-8ba6094e-2f14-4eac-bead-e623f567b1ea req-cf75b127-0927-46d5-8db9-d5a8c9785a7b service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:35.393214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-8ba6094e-2f14-4eac-bead-e623f567b1ea req-cf75b127-0927-46d5-8db9-d5a8c9785a7b service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:35.396102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 87/181] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:35 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:35.410419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-8ba6094e-2f14-4eac-bead-e623f567b1ea req-cf75b127-0927-46d5-8db9-d5a8c9785a7b service nova] Creating event network-changed:207fb2dc-4cc9-4491-b6db-03ae99699ac3 for instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 on hv-cinder-85646 Sep 08 13:54:35.417032 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-8ba6094e-2f14-4eac-bead-e623f567b1ea req-cf75b127-0927-46d5-8db9-d5a8c9785a7b service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.043095 Sep 08 13:54:35.417976 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 95/182] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:54:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:54:35.946206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f9a7a13e-ae14-4763-910c-a00b0816aeb8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:35.947160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f9a7a13e-ae14-4763-910c-a00b0816aeb8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:35.956209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9a7a13e-ae14-4763-910c-a00b0816aeb8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:35.959311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9a7a13e-ae14-4763-910c-a00b0816aeb8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:35.959311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9a7a13e-ae14-4763-910c-a00b0816aeb8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:35.959311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9a7a13e-ae14-4763-910c-a00b0816aeb8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:35.959311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9a7a13e-ae14-4763-910c-a00b0816aeb8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:35.959784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9a7a13e-ae14-4763-910c-a00b0816aeb8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:36.298538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f9a7a13e-ae14-4763-910c-a00b0816aeb8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:36.298538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f9a7a13e-ae14-4763-910c-a00b0816aeb8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.189365 Sep 08 13:54:36.298538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 88/183] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:35 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:36.495610 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-30e3120c-147e-4948-8b26-ff08ba924bf8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:36.495610 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-30e3120c-147e-4948-8b26-ff08ba924bf8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:36.495610 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-30e3120c-147e-4948-8b26-ff08ba924bf8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:36.495610 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-30e3120c-147e-4948-8b26-ff08ba924bf8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:36.497378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-30e3120c-147e-4948-8b26-ff08ba924bf8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:36.497378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-30e3120c-147e-4948-8b26-ff08ba924bf8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:36.497378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-30e3120c-147e-4948-8b26-ff08ba924bf8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:36.497378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-30e3120c-147e-4948-8b26-ff08ba924bf8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:36.572907 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-30e3120c-147e-4948-8b26-ff08ba924bf8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:36.583621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-30e3120c-147e-4948-8b26-ff08ba924bf8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:36.585048 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-30e3120c-147e-4948-8b26-ff08ba924bf8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:36.586330 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-30e3120c-147e-4948-8b26-ff08ba924bf8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1407 microversion: 2.20 time: 0.176216 Sep 08 13:54:36.586868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 96/184] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:36 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1407 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:37.155481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-be2eb501-c7d9-4649-affa-0f9a1b41cd24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:37.156152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-be2eb501-c7d9-4649-affa-0f9a1b41cd24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:37.167454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be2eb501-c7d9-4649-affa-0f9a1b41cd24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:37.172973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be2eb501-c7d9-4649-affa-0f9a1b41cd24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:37.172973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be2eb501-c7d9-4649-affa-0f9a1b41cd24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:37.172973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be2eb501-c7d9-4649-affa-0f9a1b41cd24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:37.172973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be2eb501-c7d9-4649-affa-0f9a1b41cd24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:37.173472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be2eb501-c7d9-4649-affa-0f9a1b41cd24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:37.365690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-be2eb501-c7d9-4649-affa-0f9a1b41cd24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:37.376484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-be2eb501-c7d9-4649-affa-0f9a1b41cd24 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.225198 Sep 08 13:54:37.377571 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 89/185] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:37 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:37.614324 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3ff73204-1dd7-4fd3-a89e-27d4b1adb33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:37.615545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3ff73204-1dd7-4fd3-a89e-27d4b1adb33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:37.622187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3ff73204-1dd7-4fd3-a89e-27d4b1adb33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:37.624294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3ff73204-1dd7-4fd3-a89e-27d4b1adb33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:37.624812 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3ff73204-1dd7-4fd3-a89e-27d4b1adb33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:37.625613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3ff73204-1dd7-4fd3-a89e-27d4b1adb33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:37.626071 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3ff73204-1dd7-4fd3-a89e-27d4b1adb33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:37.626719 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3ff73204-1dd7-4fd3-a89e-27d4b1adb33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:37.890407 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3ff73204-1dd7-4fd3-a89e-27d4b1adb33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:37.894877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:37.900324 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3ff73204-1dd7-4fd3-a89e-27d4b1adb33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:37.902307 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3ff73204-1dd7-4fd3-a89e-27d4b1adb33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:37.904517 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3ff73204-1dd7-4fd3-a89e-27d4b1adb33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1597 microversion: 2.20 time: 0.293402 Sep 08 13:54:37.905284 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 97/186] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:54:37 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1597 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:54:37.930996 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2ecc3688-7ab8-42fe-9f2b-b283de762117 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:37.935488 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2ecc3688-7ab8-42fe-9f2b-b283de762117 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:37.937775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ecc3688-7ab8-42fe-9f2b-b283de762117 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:37.938303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ecc3688-7ab8-42fe-9f2b-b283de762117 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:37.938768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ecc3688-7ab8-42fe-9f2b-b283de762117 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:37.939465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ecc3688-7ab8-42fe-9f2b-b283de762117 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:37.939917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ecc3688-7ab8-42fe-9f2b-b283de762117 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:37.940337 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ecc3688-7ab8-42fe-9f2b-b283de762117 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:38.024166 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2ecc3688-7ab8-42fe-9f2b-b283de762117 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.097093 Sep 08 13:54:38.024762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 90/187] 10.222.0.25 () {60 vars in 1316 bytes} [Thu Sep 8 13:54:37 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-interface => generated 284 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:54:38.415597 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-52710722-dd77-4132-bd2d-8aba95537596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:38.415597 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-52710722-dd77-4132-bd2d-8aba95537596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:38.428255 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-52710722-dd77-4132-bd2d-8aba95537596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:38.428255 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-52710722-dd77-4132-bd2d-8aba95537596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:38.428255 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-52710722-dd77-4132-bd2d-8aba95537596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:38.428255 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-52710722-dd77-4132-bd2d-8aba95537596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:38.428577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-52710722-dd77-4132-bd2d-8aba95537596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:38.428577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-52710722-dd77-4132-bd2d-8aba95537596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:38.654024 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-52710722-dd77-4132-bd2d-8aba95537596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:38.665109 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-52710722-dd77-4132-bd2d-8aba95537596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.266912 Sep 08 13:54:38.669485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 98/188] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:38 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:39.489605 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-8f3ea244-dfb8-4a89-a8fd-419803473066 req-97d5d13d-8e7e-4371-8f27-c2668f070287 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c14f6817-e3ab-4a36-ba23-590498bf1b25", "tag": "4894f431-b6f6-46b3-95db-e9eb88227e33"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:54:39.500000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-8f3ea244-dfb8-4a89-a8fd-419803473066 req-97d5d13d-8e7e-4371-8f27-c2668f070287 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:39.501661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-8f3ea244-dfb8-4a89-a8fd-419803473066 req-97d5d13d-8e7e-4371-8f27-c2668f070287 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:39.501661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-8f3ea244-dfb8-4a89-a8fd-419803473066 req-97d5d13d-8e7e-4371-8f27-c2668f070287 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:39.512864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-8f3ea244-dfb8-4a89-a8fd-419803473066 req-97d5d13d-8e7e-4371-8f27-c2668f070287 service nova] Creating event network-changed:4894f431-b6f6-46b3-95db-e9eb88227e33 for instance c14f6817-e3ab-4a36-ba23-590498bf1b25 on hv-cinder-85646 Sep 08 13:54:39.518271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-8f3ea244-dfb8-4a89-a8fd-419803473066 req-97d5d13d-8e7e-4371-8f27-c2668f070287 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032801 Sep 08 13:54:39.518935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 91/189] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:54:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:54:39.693097 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ada085af-3781-4ba6-859b-833a8ee1bc55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:39.693768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ada085af-3781-4ba6-859b-833a8ee1bc55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:39.702355 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ada085af-3781-4ba6-859b-833a8ee1bc55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:39.702355 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ada085af-3781-4ba6-859b-833a8ee1bc55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:39.702355 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ada085af-3781-4ba6-859b-833a8ee1bc55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:39.702617 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ada085af-3781-4ba6-859b-833a8ee1bc55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:39.702844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ada085af-3781-4ba6-859b-833a8ee1bc55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:39.703258 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ada085af-3781-4ba6-859b-833a8ee1bc55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:39.869724 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ada085af-3781-4ba6-859b-833a8ee1bc55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:39.877574 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ada085af-3781-4ba6-859b-833a8ee1bc55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.189845 Sep 08 13:54:39.878166 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 99/190] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:39 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:40.902755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-23b43070-e61b-4f48-b957-7c378698b91e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:40.903520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-23b43070-e61b-4f48-b957-7c378698b91e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:40.913147 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23b43070-e61b-4f48-b957-7c378698b91e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:40.913147 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23b43070-e61b-4f48-b957-7c378698b91e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:40.913147 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23b43070-e61b-4f48-b957-7c378698b91e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:40.913491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23b43070-e61b-4f48-b957-7c378698b91e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:40.913832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23b43070-e61b-4f48-b957-7c378698b91e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:40.914283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23b43070-e61b-4f48-b957-7c378698b91e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:41.051811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-23b43070-e61b-4f48-b957-7c378698b91e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:41.059393 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-23b43070-e61b-4f48-b957-7c378698b91e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.162140 Sep 08 13:54:41.059978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 92/191] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:40 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:42.081742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f52d974e-e426-41e3-96d7-43ac361d76ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:42.082350 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f52d974e-e426-41e3-96d7-43ac361d76ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:42.088236 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f52d974e-e426-41e3-96d7-43ac361d76ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:42.088705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f52d974e-e426-41e3-96d7-43ac361d76ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:42.089127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f52d974e-e426-41e3-96d7-43ac361d76ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:42.089799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f52d974e-e426-41e3-96d7-43ac361d76ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:42.090274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f52d974e-e426-41e3-96d7-43ac361d76ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:42.090989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f52d974e-e426-41e3-96d7-43ac361d76ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:42.219768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f52d974e-e426-41e3-96d7-43ac361d76ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:42.227455 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f52d974e-e426-41e3-96d7-43ac361d76ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.149663 Sep 08 13:54:42.228025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 100/192] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:42 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:43.252828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f2390cb5-528e-440a-a556-63214483c8f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:43.254057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f2390cb5-528e-440a-a556-63214483c8f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:43.256790 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=97 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:43.261940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2390cb5-528e-440a-a556-63214483c8f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:43.262809 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2390cb5-528e-440a-a556-63214483c8f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:43.263943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2390cb5-528e-440a-a556-63214483c8f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:43.267258 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2390cb5-528e-440a-a556-63214483c8f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:43.267258 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2390cb5-528e-440a-a556-63214483c8f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:43.267258 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2390cb5-528e-440a-a556-63214483c8f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:43.470742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f2390cb5-528e-440a-a556-63214483c8f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:43.485675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f2390cb5-528e-440a-a556-63214483c8f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.230187 Sep 08 13:54:43.485675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 93/193] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:43 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:44.502913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-39382b0b-5515-4402-bdb1-678e90ed2b6f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:44.503520 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-39382b0b-5515-4402-bdb1-678e90ed2b6f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:44.516506 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39382b0b-5515-4402-bdb1-678e90ed2b6f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:44.516506 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39382b0b-5515-4402-bdb1-678e90ed2b6f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:44.516506 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39382b0b-5515-4402-bdb1-678e90ed2b6f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:44.516506 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39382b0b-5515-4402-bdb1-678e90ed2b6f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:44.516882 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39382b0b-5515-4402-bdb1-678e90ed2b6f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:44.516882 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39382b0b-5515-4402-bdb1-678e90ed2b6f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:44.652802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-39382b0b-5515-4402-bdb1-678e90ed2b6f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:44.661035 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-39382b0b-5515-4402-bdb1-678e90ed2b6f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.161909 Sep 08 13:54:44.665628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 101/194] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:44 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:45.686748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2fa90277-a8aa-4291-996c-58c88e819d74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:45.687357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2fa90277-a8aa-4291-996c-58c88e819d74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:45.696837 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fa90277-a8aa-4291-996c-58c88e819d74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:45.697506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fa90277-a8aa-4291-996c-58c88e819d74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:45.698095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fa90277-a8aa-4291-996c-58c88e819d74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:45.699325 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fa90277-a8aa-4291-996c-58c88e819d74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:45.700215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fa90277-a8aa-4291-996c-58c88e819d74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:45.700644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fa90277-a8aa-4291-996c-58c88e819d74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:45.850980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2fa90277-a8aa-4291-996c-58c88e819d74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:45.853927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2fa90277-a8aa-4291-996c-58c88e819d74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.171664 Sep 08 13:54:45.855113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 94/195] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:45 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:46.879415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-67d3379b-acb4-4112-b289-7fb7cdc68829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:46.881400 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-67d3379b-acb4-4112-b289-7fb7cdc68829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:46.886819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67d3379b-acb4-4112-b289-7fb7cdc68829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:46.887705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67d3379b-acb4-4112-b289-7fb7cdc68829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:46.887705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67d3379b-acb4-4112-b289-7fb7cdc68829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:46.889031 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67d3379b-acb4-4112-b289-7fb7cdc68829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:46.889512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67d3379b-acb4-4112-b289-7fb7cdc68829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:46.890464 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67d3379b-acb4-4112-b289-7fb7cdc68829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:47.042547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-67d3379b-acb4-4112-b289-7fb7cdc68829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:47.051037 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-67d3379b-acb4-4112-b289-7fb7cdc68829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.175486 Sep 08 13:54:47.051614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 102/196] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:46 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:48.077058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-68f97a4e-b8c2-417d-ac4a-d111e9553860 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:48.077894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-68f97a4e-b8c2-417d-ac4a-d111e9553860 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:48.085059 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68f97a4e-b8c2-417d-ac4a-d111e9553860 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:48.085607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68f97a4e-b8c2-417d-ac4a-d111e9553860 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:48.092643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68f97a4e-b8c2-417d-ac4a-d111e9553860 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:48.092643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68f97a4e-b8c2-417d-ac4a-d111e9553860 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:48.092643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68f97a4e-b8c2-417d-ac4a-d111e9553860 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:48.092643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68f97a4e-b8c2-417d-ac4a-d111e9553860 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:48.093222 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:48.231229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-68f97a4e-b8c2-417d-ac4a-d111e9553860 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:48.239292 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-68f97a4e-b8c2-417d-ac4a-d111e9553860 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.166741 Sep 08 13:54:48.239865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 95/197] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:48 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:49.262094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-44298fca-71d4-42dc-a8ca-43112d4b618a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:49.262696 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-44298fca-71d4-42dc-a8ca-43112d4b618a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:49.269463 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-44298fca-71d4-42dc-a8ca-43112d4b618a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:49.269928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-44298fca-71d4-42dc-a8ca-43112d4b618a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:49.270391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-44298fca-71d4-42dc-a8ca-43112d4b618a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:49.271064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-44298fca-71d4-42dc-a8ca-43112d4b618a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:49.271503 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-44298fca-71d4-42dc-a8ca-43112d4b618a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:49.271915 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-44298fca-71d4-42dc-a8ca-43112d4b618a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:49.423263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-44298fca-71d4-42dc-a8ca-43112d4b618a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:49.430952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-44298fca-71d4-42dc-a8ca-43112d4b618a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.172749 Sep 08 13:54:49.431532 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 103/198] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:49 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:50.457920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3d2e244f-e054-4477-8f41-d75f3c1e838c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:50.458907 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3d2e244f-e054-4477-8f41-d75f3c1e838c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:50.479796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d2e244f-e054-4477-8f41-d75f3c1e838c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:50.479796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d2e244f-e054-4477-8f41-d75f3c1e838c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:50.479796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d2e244f-e054-4477-8f41-d75f3c1e838c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:50.479796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d2e244f-e054-4477-8f41-d75f3c1e838c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:50.480307 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d2e244f-e054-4477-8f41-d75f3c1e838c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:50.480307 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d2e244f-e054-4477-8f41-d75f3c1e838c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:50.634184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3d2e244f-e054-4477-8f41-d75f3c1e838c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:50.643153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3d2e244f-e054-4477-8f41-d75f3c1e838c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.191608 Sep 08 13:54:50.643734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 96/199] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:50 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:51.666649 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5ca7e05d-8b6b-4b4f-ad6e-7ecd31d8d1d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:51.667260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5ca7e05d-8b6b-4b4f-ad6e-7ecd31d8d1d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:51.674965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5ca7e05d-8b6b-4b4f-ad6e-7ecd31d8d1d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:51.676535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5ca7e05d-8b6b-4b4f-ad6e-7ecd31d8d1d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:51.676535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5ca7e05d-8b6b-4b4f-ad6e-7ecd31d8d1d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:51.677518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5ca7e05d-8b6b-4b4f-ad6e-7ecd31d8d1d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:51.678125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5ca7e05d-8b6b-4b4f-ad6e-7ecd31d8d1d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:51.679193 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5ca7e05d-8b6b-4b4f-ad6e-7ecd31d8d1d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:51.834443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5ca7e05d-8b6b-4b4f-ad6e-7ecd31d8d1d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:51.836981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5ca7e05d-8b6b-4b4f-ad6e-7ecd31d8d1d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.174412 Sep 08 13:54:51.837574 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 104/200] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:51 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:52.864969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-21e720ca-28c2-4381-af38-c130ea4d19ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:52.865577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-21e720ca-28c2-4381-af38-c130ea4d19ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:52.872244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-21e720ca-28c2-4381-af38-c130ea4d19ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:52.874432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-21e720ca-28c2-4381-af38-c130ea4d19ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:52.874961 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-21e720ca-28c2-4381-af38-c130ea4d19ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:52.875682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-21e720ca-28c2-4381-af38-c130ea4d19ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:52.876142 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-21e720ca-28c2-4381-af38-c130ea4d19ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:52.876562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-21e720ca-28c2-4381-af38-c130ea4d19ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:53.010541 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-21e720ca-28c2-4381-af38-c130ea4d19ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:53.042901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-21e720ca-28c2-4381-af38-c130ea4d19ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.181836 Sep 08 13:54:53.048324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 97/201] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:52 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:54.066975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ef087000-8bf0-45df-8f0e-e1fa565b2bfe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:54.068949 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ef087000-8bf0-45df-8f0e-e1fa565b2bfe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:54.076347 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef087000-8bf0-45df-8f0e-e1fa565b2bfe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:54.077730 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef087000-8bf0-45df-8f0e-e1fa565b2bfe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:54.078367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef087000-8bf0-45df-8f0e-e1fa565b2bfe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:54.079147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef087000-8bf0-45df-8f0e-e1fa565b2bfe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:54.082362 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef087000-8bf0-45df-8f0e-e1fa565b2bfe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:54.082652 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef087000-8bf0-45df-8f0e-e1fa565b2bfe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:54.248116 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ef087000-8bf0-45df-8f0e-e1fa565b2bfe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:54.256416 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ef087000-8bf0-45df-8f0e-e1fa565b2bfe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.193869 Sep 08 13:54:54.256981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 105/202] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:54 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:54.702009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a4e06159-e025-4b91-9530-b5d6d77f817b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:54.702009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a4e06159-e025-4b91-9530-b5d6d77f817b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:54.706813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4e06159-e025-4b91-9530-b5d6d77f817b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:54.707344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4e06159-e025-4b91-9530-b5d6d77f817b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:54.708661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4e06159-e025-4b91-9530-b5d6d77f817b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:54.709390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4e06159-e025-4b91-9530-b5d6d77f817b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:54.709849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4e06159-e025-4b91-9530-b5d6d77f817b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:54.710309 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4e06159-e025-4b91-9530-b5d6d77f817b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:54.739806 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a4e06159-e025-4b91-9530-b5d6d77f817b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 13:54:55.126058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a4e06159-e025-4b91-9530-b5d6d77f817b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "DELETE /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 204 len: 0 microversion: 2.1 time: 0.432249 Sep 08 13:54:55.126880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 98/203] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 13:54:54 2022] DELETE /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 13:54:55.135370 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f4edb6a8-eba1-496f-9c6c-19feee7ff92b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:55.135983 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f4edb6a8-eba1-496f-9c6c-19feee7ff92b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:55.143866 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f4edb6a8-eba1-496f-9c6c-19feee7ff92b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:55.144478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f4edb6a8-eba1-496f-9c6c-19feee7ff92b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:55.144968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f4edb6a8-eba1-496f-9c6c-19feee7ff92b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:55.145857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f4edb6a8-eba1-496f-9c6c-19feee7ff92b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:55.146349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f4edb6a8-eba1-496f-9c6c-19feee7ff92b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:55.146804 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f4edb6a8-eba1-496f-9c6c-19feee7ff92b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:55.281428 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ce9a1c92-c1c7-4e3e-bee5-f091809cd3df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:55.282196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ce9a1c92-c1c7-4e3e-bee5-f091809cd3df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:55.296797 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f4edb6a8-eba1-496f-9c6c-19feee7ff92b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:55.297048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ce9a1c92-c1c7-4e3e-bee5-f091809cd3df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:55.297048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ce9a1c92-c1c7-4e3e-bee5-f091809cd3df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:55.297048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ce9a1c92-c1c7-4e3e-bee5-f091809cd3df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:55.297048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ce9a1c92-c1c7-4e3e-bee5-f091809cd3df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:55.297390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ce9a1c92-c1c7-4e3e-bee5-f091809cd3df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:55.297390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ce9a1c92-c1c7-4e3e-bee5-f091809cd3df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:55.297579 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f4edb6a8-eba1-496f-9c6c-19feee7ff92b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1554 microversion: 2.1 time: 0.165189 Sep 08 13:54:55.297579 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 106/204] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:55 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1554 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:55.434216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ce9a1c92-c1c7-4e3e-bee5-f091809cd3df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:55.442174 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ce9a1c92-c1c7-4e3e-bee5-f091809cd3df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.165814 Sep 08 13:54:55.443195 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 99/205] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:55 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:56.321161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7f6f04e0-81f0-4ab4-b007-839e174d290e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:56.321772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7f6f04e0-81f0-4ab4-b007-839e174d290e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:56.330839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7f6f04e0-81f0-4ab4-b007-839e174d290e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:56.331586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7f6f04e0-81f0-4ab4-b007-839e174d290e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:56.332200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7f6f04e0-81f0-4ab4-b007-839e174d290e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:56.333440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7f6f04e0-81f0-4ab4-b007-839e174d290e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:56.334051 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7f6f04e0-81f0-4ab4-b007-839e174d290e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:56.334493 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7f6f04e0-81f0-4ab4-b007-839e174d290e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:56.475538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-52c62df1-54b4-4df9-8be1-edb214731533 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:56.476349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-52c62df1-54b4-4df9-8be1-edb214731533 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:56.481803 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7f6f04e0-81f0-4ab4-b007-839e174d290e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:56.482872 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-52c62df1-54b4-4df9-8be1-edb214731533 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:56.483610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-52c62df1-54b4-4df9-8be1-edb214731533 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:56.484214 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-52c62df1-54b4-4df9-8be1-edb214731533 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:56.484953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-52c62df1-54b4-4df9-8be1-edb214731533 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:56.485407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-52c62df1-54b4-4df9-8be1-edb214731533 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:56.485837 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-52c62df1-54b4-4df9-8be1-edb214731533 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:56.492210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7f6f04e0-81f0-4ab4-b007-839e174d290e tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1554 microversion: 2.1 time: 0.175580 Sep 08 13:54:56.493024 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 107/206] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:56 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1554 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:56.632160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-52c62df1-54b4-4df9-8be1-edb214731533 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:56.640374 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-52c62df1-54b4-4df9-8be1-edb214731533 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.168677 Sep 08 13:54:56.641631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 100/207] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:56 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:57.520105 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6a11c3b4-a54f-489b-918b-3ea0ba417df0 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:57.521166 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6a11c3b4-a54f-489b-918b-3ea0ba417df0 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:57.527481 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a11c3b4-a54f-489b-918b-3ea0ba417df0 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:57.528028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a11c3b4-a54f-489b-918b-3ea0ba417df0 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:57.528604 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a11c3b4-a54f-489b-918b-3ea0ba417df0 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:57.529288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a11c3b4-a54f-489b-918b-3ea0ba417df0 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:57.529732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a11c3b4-a54f-489b-918b-3ea0ba417df0 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:57.530159 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a11c3b4-a54f-489b-918b-3ea0ba417df0 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:57.664228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-80cf4264-faf3-4567-99a8-90d30612f157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:57.665390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-80cf4264-faf3-4567-99a8-90d30612f157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:57.671803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-80cf4264-faf3-4567-99a8-90d30612f157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:57.675279 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6a11c3b4-a54f-489b-918b-3ea0ba417df0 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:57.677902 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-80cf4264-faf3-4567-99a8-90d30612f157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:57.678459 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-80cf4264-faf3-4567-99a8-90d30612f157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:57.679198 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-80cf4264-faf3-4567-99a8-90d30612f157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:57.679662 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-80cf4264-faf3-4567-99a8-90d30612f157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:57.680087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-80cf4264-faf3-4567-99a8-90d30612f157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:57.683729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6a11c3b4-a54f-489b-918b-3ea0ba417df0 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1554 microversion: 2.1 time: 0.171060 Sep 08 13:54:57.684366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 108/208] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:57 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1554 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:57.810867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-80cf4264-faf3-4567-99a8-90d30612f157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:57.819461 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-80cf4264-faf3-4567-99a8-90d30612f157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.159072 Sep 08 13:54:57.820042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 101/209] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:57 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:54:58.708425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c1f9ad92-1430-4b77-9261-1b583b5567e5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:58.709431 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c1f9ad92-1430-4b77-9261-1b583b5567e5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:58.716637 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c1f9ad92-1430-4b77-9261-1b583b5567e5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:58.717191 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c1f9ad92-1430-4b77-9261-1b583b5567e5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:58.717554 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c1f9ad92-1430-4b77-9261-1b583b5567e5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:58.718305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c1f9ad92-1430-4b77-9261-1b583b5567e5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:58.718750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c1f9ad92-1430-4b77-9261-1b583b5567e5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:58.719884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c1f9ad92-1430-4b77-9261-1b583b5567e5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:58.843252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b3fd17b1-bd4f-4d08-826b-2fca1faa0b2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:58.844497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b3fd17b1-bd4f-4d08-826b-2fca1faa0b2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:58.850968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b3fd17b1-bd4f-4d08-826b-2fca1faa0b2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:58.851486 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b3fd17b1-bd4f-4d08-826b-2fca1faa0b2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:58.851923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b3fd17b1-bd4f-4d08-826b-2fca1faa0b2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:58.852626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b3fd17b1-bd4f-4d08-826b-2fca1faa0b2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:58.853082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b3fd17b1-bd4f-4d08-826b-2fca1faa0b2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:58.853504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b3fd17b1-bd4f-4d08-826b-2fca1faa0b2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:58.871152 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c1f9ad92-1430-4b77-9261-1b583b5567e5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:58.880773 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c1f9ad92-1430-4b77-9261-1b583b5567e5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1554 microversion: 2.1 time: 0.176858 Sep 08 13:54:58.883482 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 109/210] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:58 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1554 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:58.990955 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b3fd17b1-bd4f-4d08-826b-2fca1faa0b2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:54:58.998849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b3fd17b1-bd4f-4d08-826b-2fca1faa0b2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.159768 Sep 08 13:54:58.999412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 102/211] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:58 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:54:59.905152 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-39628d4c-91d7-4c9b-a63b-3521ffde0284 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:54:59.905763 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-39628d4c-91d7-4c9b-a63b-3521ffde0284 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:54:59.912125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39628d4c-91d7-4c9b-a63b-3521ffde0284 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:59.912611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39628d4c-91d7-4c9b-a63b-3521ffde0284 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:59.913041 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39628d4c-91d7-4c9b-a63b-3521ffde0284 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:54:59.913725 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39628d4c-91d7-4c9b-a63b-3521ffde0284 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:54:59.914197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39628d4c-91d7-4c9b-a63b-3521ffde0284 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:54:59.914619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39628d4c-91d7-4c9b-a63b-3521ffde0284 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:00.029541 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0ad7f466-e522-4d2a-a02d-a7ee454b3ea8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:00.029541 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0ad7f466-e522-4d2a-a02d-a7ee454b3ea8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:00.031734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ad7f466-e522-4d2a-a02d-a7ee454b3ea8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:00.032394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ad7f466-e522-4d2a-a02d-a7ee454b3ea8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:00.032988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ad7f466-e522-4d2a-a02d-a7ee454b3ea8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:00.034062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ad7f466-e522-4d2a-a02d-a7ee454b3ea8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:00.034562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ad7f466-e522-4d2a-a02d-a7ee454b3ea8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:00.035134 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ad7f466-e522-4d2a-a02d-a7ee454b3ea8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:00.070629 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-39628d4c-91d7-4c9b-a63b-3521ffde0284 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:00.081510 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-39628d4c-91d7-4c9b-a63b-3521ffde0284 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1554 microversion: 2.1 time: 0.181130 Sep 08 13:55:00.082604 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 110/212] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:54:59 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1554 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:00.196318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0ad7f466-e522-4d2a-a02d-a7ee454b3ea8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:00.203121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0ad7f466-e522-4d2a-a02d-a7ee454b3ea8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.183999 Sep 08 13:55:00.203754 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 103/213] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:00 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:55:01.107571 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d13624aa-0422-48c9-8115-13d1088b8580 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:01.108181 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d13624aa-0422-48c9-8115-13d1088b8580 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:01.114213 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d13624aa-0422-48c9-8115-13d1088b8580 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:01.114698 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d13624aa-0422-48c9-8115-13d1088b8580 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:01.115130 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d13624aa-0422-48c9-8115-13d1088b8580 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:01.115820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d13624aa-0422-48c9-8115-13d1088b8580 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:01.116268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d13624aa-0422-48c9-8115-13d1088b8580 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:01.116684 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d13624aa-0422-48c9-8115-13d1088b8580 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:01.227351 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2f8a075a-1a23-41fe-a05c-a893e85742c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:01.227958 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2f8a075a-1a23-41fe-a05c-a893e85742c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:01.234965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f8a075a-1a23-41fe-a05c-a893e85742c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:01.235455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f8a075a-1a23-41fe-a05c-a893e85742c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:01.235888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f8a075a-1a23-41fe-a05c-a893e85742c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:01.236588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f8a075a-1a23-41fe-a05c-a893e85742c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:01.237058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f8a075a-1a23-41fe-a05c-a893e85742c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:01.237463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f8a075a-1a23-41fe-a05c-a893e85742c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:01.273001 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d13624aa-0422-48c9-8115-13d1088b8580 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:01.283444 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d13624aa-0422-48c9-8115-13d1088b8580 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1554 microversion: 2.1 time: 0.181330 Sep 08 13:55:01.285139 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 111/214] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:01 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1554 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:01.387652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2f8a075a-1a23-41fe-a05c-a893e85742c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:01.396169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2f8a075a-1a23-41fe-a05c-a893e85742c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.172587 Sep 08 13:55:01.396947 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 104/215] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:01 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:02.308701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7e2aa482-c725-4180-a4f7-72b5bf8d84c6 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:02.309627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7e2aa482-c725-4180-a4f7-72b5bf8d84c6 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:02.315782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e2aa482-c725-4180-a4f7-72b5bf8d84c6 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:02.316336 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e2aa482-c725-4180-a4f7-72b5bf8d84c6 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:02.317002 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e2aa482-c725-4180-a4f7-72b5bf8d84c6 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:02.317925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e2aa482-c725-4180-a4f7-72b5bf8d84c6 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:02.318603 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e2aa482-c725-4180-a4f7-72b5bf8d84c6 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:02.319020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e2aa482-c725-4180-a4f7-72b5bf8d84c6 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:02.424932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ccc7f7e3-cbbb-487b-895c-a4fddecc4caf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:02.425548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ccc7f7e3-cbbb-487b-895c-a4fddecc4caf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:02.432084 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ccc7f7e3-cbbb-487b-895c-a4fddecc4caf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:02.432577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ccc7f7e3-cbbb-487b-895c-a4fddecc4caf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:02.433021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ccc7f7e3-cbbb-487b-895c-a4fddecc4caf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:02.433721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ccc7f7e3-cbbb-487b-895c-a4fddecc4caf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:02.434342 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ccc7f7e3-cbbb-487b-895c-a4fddecc4caf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:02.434779 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ccc7f7e3-cbbb-487b-895c-a4fddecc4caf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:02.600075 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ccc7f7e3-cbbb-487b-895c-a4fddecc4caf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:02.608101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ccc7f7e3-cbbb-487b-895c-a4fddecc4caf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.186981 Sep 08 13:55:02.608690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 105/216] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:02 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:02.754632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7e2aa482-c725-4180-a4f7-72b5bf8d84c6 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:02.763058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7e2aa482-c725-4180-a4f7-72b5bf8d84c6 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1554 microversion: 2.1 time: 0.458849 Sep 08 13:55:02.763637 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 112/217] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:02 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1554 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:03.025434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-55d1a2a1-77da-4e2b-a348-877390a0d4d8 req-42e433fe-5fe0-4c69-ae10-43e53bb6bb24 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "358e292a-9e96-4380-8ea5-3ed00372b4c3", "name": "network-vif-deleted", "tag": "a0d989dd-29c8-414f-a850-15a4e3aea7a8"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:55:03.035721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-55d1a2a1-77da-4e2b-a348-877390a0d4d8 req-42e433fe-5fe0-4c69-ae10-43e53bb6bb24 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:03.036219 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-55d1a2a1-77da-4e2b-a348-877390a0d4d8 req-42e433fe-5fe0-4c69-ae10-43e53bb6bb24 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:03.036648 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-55d1a2a1-77da-4e2b-a348-877390a0d4d8 req-42e433fe-5fe0-4c69-ae10-43e53bb6bb24 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:03.050038 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-55d1a2a1-77da-4e2b-a348-877390a0d4d8 req-42e433fe-5fe0-4c69-ae10-43e53bb6bb24 service nova] Creating event network-vif-deleted:a0d989dd-29c8-414f-a850-15a4e3aea7a8 for instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 on hv-cinder-85646 Sep 08 13:55:03.055612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-55d1a2a1-77da-4e2b-a348-877390a0d4d8 req-42e433fe-5fe0-4c69-ae10-43e53bb6bb24 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033745 Sep 08 13:55:03.056749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 106/218] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:55:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:55:03.631557 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-195fab01-19de-410c-a050-36b523da9b56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:03.632300 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-195fab01-19de-410c-a050-36b523da9b56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:03.634345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=43 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:03.639148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-195fab01-19de-410c-a050-36b523da9b56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:03.639799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-195fab01-19de-410c-a050-36b523da9b56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:03.640238 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-195fab01-19de-410c-a050-36b523da9b56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:03.642117 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-195fab01-19de-410c-a050-36b523da9b56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:03.642117 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-195fab01-19de-410c-a050-36b523da9b56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:03.642573 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-195fab01-19de-410c-a050-36b523da9b56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:03.782498 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-195fab01-19de-410c-a050-36b523da9b56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:03.788636 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c54ee9ce-a77d-47d2-9a2d-74e84d5ed6f5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:03.789339 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c54ee9ce-a77d-47d2-9a2d-74e84d5ed6f5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:03.790445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-195fab01-19de-410c-a050-36b523da9b56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.162843 Sep 08 13:55:03.791018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 113/219] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:03 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:03.795450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c54ee9ce-a77d-47d2-9a2d-74e84d5ed6f5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:03.795937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c54ee9ce-a77d-47d2-9a2d-74e84d5ed6f5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:03.796372 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c54ee9ce-a77d-47d2-9a2d-74e84d5ed6f5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:03.797062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c54ee9ce-a77d-47d2-9a2d-74e84d5ed6f5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:03.797524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c54ee9ce-a77d-47d2-9a2d-74e84d5ed6f5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:03.798049 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c54ee9ce-a77d-47d2-9a2d-74e84d5ed6f5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:03.862787 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c54ee9ce-a77d-47d2-9a2d-74e84d5ed6f5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:03.871017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c54ee9ce-a77d-47d2-9a2d-74e84d5ed6f5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1512 microversion: 2.1 time: 0.087113 Sep 08 13:55:03.871725 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 107/220] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:03 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1512 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:55:04.813735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bafbcf59-3e12-4cc7-b6e3-57c99969aa82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:04.814373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bafbcf59-3e12-4cc7-b6e3-57c99969aa82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:04.820784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bafbcf59-3e12-4cc7-b6e3-57c99969aa82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:04.821266 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bafbcf59-3e12-4cc7-b6e3-57c99969aa82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:04.821700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bafbcf59-3e12-4cc7-b6e3-57c99969aa82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:04.822420 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bafbcf59-3e12-4cc7-b6e3-57c99969aa82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:04.822874 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bafbcf59-3e12-4cc7-b6e3-57c99969aa82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:04.823295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bafbcf59-3e12-4cc7-b6e3-57c99969aa82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:04.841180 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:04.899819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-af5feb30-8a96-4abe-b40b-c92416ab129f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:04.900328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-af5feb30-8a96-4abe-b40b-c92416ab129f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:04.906966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af5feb30-8a96-4abe-b40b-c92416ab129f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:04.907464 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af5feb30-8a96-4abe-b40b-c92416ab129f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:04.907906 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af5feb30-8a96-4abe-b40b-c92416ab129f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:04.908857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af5feb30-8a96-4abe-b40b-c92416ab129f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:04.909144 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af5feb30-8a96-4abe-b40b-c92416ab129f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:04.909577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af5feb30-8a96-4abe-b40b-c92416ab129f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:04.959412 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-bafbcf59-3e12-4cc7-b6e3-57c99969aa82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:04.973998 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bafbcf59-3e12-4cc7-b6e3-57c99969aa82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.159606 Sep 08 13:55:04.973998 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 114/221] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:04 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:04.980081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-af5feb30-8a96-4abe-b40b-c92416ab129f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:04.988406 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-af5feb30-8a96-4abe-b40b-c92416ab129f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1512 microversion: 2.1 time: 0.092518 Sep 08 13:55:04.988966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 108/222] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:04 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1512 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:05.991484 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7023f90d-01d4-45d2-b910-5e5f7e8a7369 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:05.992072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7023f90d-01d4-45d2-b910-5e5f7e8a7369 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:05.997999 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7023f90d-01d4-45d2-b910-5e5f7e8a7369 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:05.998525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7023f90d-01d4-45d2-b910-5e5f7e8a7369 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:05.998957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7023f90d-01d4-45d2-b910-5e5f7e8a7369 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:05.999759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7023f90d-01d4-45d2-b910-5e5f7e8a7369 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:06.000218 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7023f90d-01d4-45d2-b910-5e5f7e8a7369 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:06.000639 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7023f90d-01d4-45d2-b910-5e5f7e8a7369 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:06.012983 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cc7a9c34-a62f-42a0-bc7d-99214b19a327 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:06.013606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cc7a9c34-a62f-42a0-bc7d-99214b19a327 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:06.030162 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cc7a9c34-a62f-42a0-bc7d-99214b19a327 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:06.030779 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cc7a9c34-a62f-42a0-bc7d-99214b19a327 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:06.031100 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cc7a9c34-a62f-42a0-bc7d-99214b19a327 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:06.031966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cc7a9c34-a62f-42a0-bc7d-99214b19a327 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:06.032371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cc7a9c34-a62f-42a0-bc7d-99214b19a327 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:06.033424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cc7a9c34-a62f-42a0-bc7d-99214b19a327 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:06.098094 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cc7a9c34-a62f-42a0-bc7d-99214b19a327 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:06.106587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cc7a9c34-a62f-42a0-bc7d-99214b19a327 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1346 microversion: 2.1 time: 0.098261 Sep 08 13:55:06.107446 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 109/223] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:06 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1346 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:55:06.151057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7023f90d-01d4-45d2-b910-5e5f7e8a7369 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:06.159497 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7023f90d-01d4-45d2-b910-5e5f7e8a7369 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1520 microversion: 2.1 time: 0.171876 Sep 08 13:55:06.160058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 115/224] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:05 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1520 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:07.128984 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b129a033-ba04-4163-bba3-13adc51a31cb tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:07.129596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b129a033-ba04-4163-bba3-13adc51a31cb tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:07.136202 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b129a033-ba04-4163-bba3-13adc51a31cb tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:07.136650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b129a033-ba04-4163-bba3-13adc51a31cb tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:07.137083 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b129a033-ba04-4163-bba3-13adc51a31cb tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:07.137902 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b129a033-ba04-4163-bba3-13adc51a31cb tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:07.138422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b129a033-ba04-4163-bba3-13adc51a31cb tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:07.138861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b129a033-ba04-4163-bba3-13adc51a31cb tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:07.182388 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-239d3c12-f8f8-44b6-91eb-7e62887cecc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:07.183036 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-239d3c12-f8f8-44b6-91eb-7e62887cecc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:07.189846 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-239d3c12-f8f8-44b6-91eb-7e62887cecc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:07.190533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-239d3c12-f8f8-44b6-91eb-7e62887cecc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:07.193644 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-239d3c12-f8f8-44b6-91eb-7e62887cecc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:07.193644 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-239d3c12-f8f8-44b6-91eb-7e62887cecc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:07.193644 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-239d3c12-f8f8-44b6-91eb-7e62887cecc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:07.193644 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-239d3c12-f8f8-44b6-91eb-7e62887cecc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:07.203618 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b129a033-ba04-4163-bba3-13adc51a31cb tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:07.207526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:07.213430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b129a033-ba04-4163-bba3-13adc51a31cb tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1346 microversion: 2.1 time: 0.088484 Sep 08 13:55:07.214017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 110/225] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:07 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1346 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:07.330522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-239d3c12-f8f8-44b6-91eb-7e62887cecc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:07.340141 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-239d3c12-f8f8-44b6-91eb-7e62887cecc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1500 microversion: 2.1 time: 0.161626 Sep 08 13:55:07.340989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 116/226] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:07 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1500 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:55:07.363614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d31e8e5c-9706-4feb-b8c5-26f856f89614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:07.367384 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d31e8e5c-9706-4feb-b8c5-26f856f89614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:07.370595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d31e8e5c-9706-4feb-b8c5-26f856f89614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:07.371067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d31e8e5c-9706-4feb-b8c5-26f856f89614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:07.371521 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d31e8e5c-9706-4feb-b8c5-26f856f89614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:07.372395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d31e8e5c-9706-4feb-b8c5-26f856f89614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:07.372658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d31e8e5c-9706-4feb-b8c5-26f856f89614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:07.373088 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d31e8e5c-9706-4feb-b8c5-26f856f89614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:07.512460 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d31e8e5c-9706-4feb-b8c5-26f856f89614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:07.519221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d31e8e5c-9706-4feb-b8c5-26f856f89614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1516 microversion: 2.1 time: 0.159151 Sep 08 13:55:07.522395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 111/227] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:07 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1516 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:08.236135 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a39292c5-b511-49c7-96ef-68bbc33bb2ae tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:08.236752 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a39292c5-b511-49c7-96ef-68bbc33bb2ae tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:08.243586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a39292c5-b511-49c7-96ef-68bbc33bb2ae tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:08.244158 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a39292c5-b511-49c7-96ef-68bbc33bb2ae tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:08.244677 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a39292c5-b511-49c7-96ef-68bbc33bb2ae tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:08.245381 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a39292c5-b511-49c7-96ef-68bbc33bb2ae tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:08.245836 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a39292c5-b511-49c7-96ef-68bbc33bb2ae tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:08.246295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a39292c5-b511-49c7-96ef-68bbc33bb2ae tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:08.283324 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a39292c5-b511-49c7-96ef-68bbc33bb2ae tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:08.283324 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a39292c5-b511-49c7-96ef-68bbc33bb2ae tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:08.283324 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a39292c5-b511-49c7-96ef-68bbc33bb2ae tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:08.283536 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-a39292c5-b511-49c7-96ef-68bbc33bb2ae tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lazy-loading 'fault' on Instance uuid 358e292a-9e96-4380-8ea5-3ed00372b4c3 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 13:55:08.330223 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a39292c5-b511-49c7-96ef-68bbc33bb2ae tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:08.338629 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a39292c5-b511-49c7-96ef-68bbc33bb2ae tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1351 microversion: 2.1 time: 0.106428 Sep 08 13:55:08.339196 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 117/228] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:08 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1351 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:09.362257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b66dc3dc-4732-4ac5-89a7-a7d4116e07a7 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:09.363544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b66dc3dc-4732-4ac5-89a7-a7d4116e07a7 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:09.370397 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b66dc3dc-4732-4ac5-89a7-a7d4116e07a7 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:09.371076 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b66dc3dc-4732-4ac5-89a7-a7d4116e07a7 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:09.371766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b66dc3dc-4732-4ac5-89a7-a7d4116e07a7 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:09.377622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b66dc3dc-4732-4ac5-89a7-a7d4116e07a7 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:09.378103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b66dc3dc-4732-4ac5-89a7-a7d4116e07a7 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:09.378614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b66dc3dc-4732-4ac5-89a7-a7d4116e07a7 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:09.409103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b66dc3dc-4732-4ac5-89a7-a7d4116e07a7 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:09.409581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b66dc3dc-4732-4ac5-89a7-a7d4116e07a7 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:09.410549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b66dc3dc-4732-4ac5-89a7-a7d4116e07a7 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:09.410999 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-b66dc3dc-4732-4ac5-89a7-a7d4116e07a7 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lazy-loading 'fault' on Instance uuid 358e292a-9e96-4380-8ea5-3ed00372b4c3 {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 13:55:09.456237 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b66dc3dc-4732-4ac5-89a7-a7d4116e07a7 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:09.464492 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b66dc3dc-4732-4ac5-89a7-a7d4116e07a7 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1351 microversion: 2.1 time: 0.106115 Sep 08 13:55:09.465058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 112/229] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:09 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1351 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:10.487139 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-febfa17c-181b-419d-8c0e-e517e44dd222 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:10.487749 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-febfa17c-181b-419d-8c0e-e517e44dd222 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:10.495031 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-febfa17c-181b-419d-8c0e-e517e44dd222 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:10.495546 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-febfa17c-181b-419d-8c0e-e517e44dd222 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:10.495943 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-febfa17c-181b-419d-8c0e-e517e44dd222 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:10.496634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-febfa17c-181b-419d-8c0e-e517e44dd222 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:10.497092 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-febfa17c-181b-419d-8c0e-e517e44dd222 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:10.497511 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-febfa17c-181b-419d-8c0e-e517e44dd222 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:10.529843 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-febfa17c-181b-419d-8c0e-e517e44dd222 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:10.530526 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-febfa17c-181b-419d-8c0e-e517e44dd222 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:10.531047 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-febfa17c-181b-419d-8c0e-e517e44dd222 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:10.531592 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-febfa17c-181b-419d-8c0e-e517e44dd222 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lazy-loading 'fault' on Instance uuid 358e292a-9e96-4380-8ea5-3ed00372b4c3 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 13:55:10.572769 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-febfa17c-181b-419d-8c0e-e517e44dd222 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8294dcc1178045abbcf7a497564eb122', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5775050045e749b1bdb26dddcce9a8ff', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:10.581179 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-febfa17c-181b-419d-8c0e-e517e44dd222 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 200 len: 1351 microversion: 2.1 time: 0.097873 Sep 08 13:55:10.581776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 118/230] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:10 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 1351 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:11.603739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fab5b51c-50cc-4c52-8997-20479aa97e7f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:11.604349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fab5b51c-50cc-4c52-8997-20479aa97e7f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:11.611231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fab5b51c-50cc-4c52-8997-20479aa97e7f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:11.611730 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fab5b51c-50cc-4c52-8997-20479aa97e7f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:11.612168 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fab5b51c-50cc-4c52-8997-20479aa97e7f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:11.613090 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fab5b51c-50cc-4c52-8997-20479aa97e7f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:11.613585 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fab5b51c-50cc-4c52-8997-20479aa97e7f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:11.613987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fab5b51c-50cc-4c52-8997-20479aa97e7f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:11.632830 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-fab5b51c-50cc-4c52-8997-20479aa97e7f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] HTTP exception thrown: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fab5b51c-50cc-4c52-8997-20479aa97e7f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Request method failure captured: Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 HTTP/1.1 Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPolVQrFmzJ1tP4tqIdAFktUoN_I-dkkz77om_IARYStbSsI7lAhhXf3o6yPalI8qFGff3S2naq5_Obm3YPQDrUDoB0Jyv9ACMCFtuErncM6uQV9c4D1tl8ur2B4mC7EcT7Io5CR0SrYMzlnBJK25Kv4EQnhwNWkmpNzX8ls27UnNU Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 5775050045e749b1bdb26dddcce9a8ff Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-DeleteServersTestJSON-2088877436 Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 13:55:11.652500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 13:55:11.654439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_5775050045e749b1bdb26dddcce9a8ff"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5775050045e749b1bdb26dddcce9a8ff"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5775050045e749b1bdb26dddcce9a8ff"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/5775050045e749b1bdb26dddcce9a8ff"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:55:11.654439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-DeleteServersTestJSON-2088877436 Sep 08 13:55:11.654439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 5775050045e749b1bdb26dddcce9a8ff Sep 08 13:55:11.654439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-DeleteServersTestJSON-2088877436 Sep 08 13:55:11.654439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-DeleteServersTestJSON-2088877436-project Sep 08 13:55:11.654439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 13:55:11.654439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 13:55:11.654439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 8294dcc1178045abbcf7a497564eb122 Sep 08 13:55:11.654439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-DeleteServersTestJSON-2088877436-project Sep 08 13:55:11.654439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 13:55:11.654439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:11.654439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': '358e292a-9e96-4380-8ea5-3ed00372b4c3'} Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:55:11.655321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:55:11.656468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:55:11.656468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 13:55:11.656468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 13:55:11.656468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 13:55:11.656468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 13:55:11.656468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 13:55:11.656468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 13:55:11.656468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:11.656468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:55:11.656468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:55:11.656468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:55:11.656468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:55:11.656468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:55:11.656468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:55:11.656468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:55:11.656468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:55:11.656468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:11.656468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:55:11.657528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fab5b51c-50cc-4c52-8997-20479aa97e7f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Returning 404 to user: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:55:11.657528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fab5b51c-50cc-4c52-8997-20479aa97e7f tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 404 len: 111 microversion: 2.1 time: 0.054352 Sep 08 13:55:11.657528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 113/231] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:11 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 13:55:11.796530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b8de7412-e0cf-41fe-be58-097d9303b50b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:11.798821 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b8de7412-e0cf-41fe-be58-097d9303b50b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:11.806463 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b8de7412-e0cf-41fe-be58-097d9303b50b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:11.807734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b8de7412-e0cf-41fe-be58-097d9303b50b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:11.808170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b8de7412-e0cf-41fe-be58-097d9303b50b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:11.810820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b8de7412-e0cf-41fe-be58-097d9303b50b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:11.810820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b8de7412-e0cf-41fe-be58-097d9303b50b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:11.810820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b8de7412-e0cf-41fe-be58-097d9303b50b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:11.825159 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-b8de7412-e0cf-41fe-be58-097d9303b50b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] HTTP exception thrown: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b8de7412-e0cf-41fe-be58-097d9303b50b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Request method failure captured: Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3/os-volume_attachments HTTP/1.1 Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPolVQrFmzJ1tP4tqIdAFktUoN_I-dkkz77om_IARYStbSsI7lAhhXf3o6yPalI8qFGff3S2naq5_Obm3YPQDrUDoB0Jyv9ACMCFtuErncM6uQV9c4D1tl8ur2B4mC7EcT7Io5CR0SrYMzlnBJK25Kv4EQnhwNWkmpNzX8ls27UnNU Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 5775050045e749b1bdb26dddcce9a8ff Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-DeleteServersTestJSON-2088877436 Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 13:55:11.829630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 13:55:11.831131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_5775050045e749b1bdb26dddcce9a8ff"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5775050045e749b1bdb26dddcce9a8ff"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5775050045e749b1bdb26dddcce9a8ff"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/5775050045e749b1bdb26dddcce9a8ff"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:55:11.831131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-DeleteServersTestJSON-2088877436 Sep 08 13:55:11.831131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 5775050045e749b1bdb26dddcce9a8ff Sep 08 13:55:11.831131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-DeleteServersTestJSON-2088877436 Sep 08 13:55:11.831131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-DeleteServersTestJSON-2088877436-project Sep 08 13:55:11.831131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 13:55:11.831131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 13:55:11.831131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 8294dcc1178045abbcf7a497564eb122 Sep 08 13:55:11.831131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-DeleteServersTestJSON-2088877436-project Sep 08 13:55:11.831131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 13:55:11.831131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:11.831131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'server_id': '358e292a-9e96-4380-8ea5-3ed00372b4c3'} Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:55:11.831989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:11.833341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:55:11.834752 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b8de7412-e0cf-41fe-be58-097d9303b50b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Returning 404 to user: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:55:11.834752 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b8de7412-e0cf-41fe-be58-097d9303b50b tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.038729 Sep 08 13:55:11.834752 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 119/232] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 13:55:11 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3/os-volume_attachments => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 13:55:17.135546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a902ce09-6068-4964-9502-3518c9690fc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:55:17.137829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a902ce09-6068-4964-9502-3518c9690fc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:17.141623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=49,nova_api:UPDATE=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:17.146932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a902ce09-6068-4964-9502-3518c9690fc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:17.154062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a902ce09-6068-4964-9502-3518c9690fc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:17.154062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a902ce09-6068-4964-9502-3518c9690fc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:17.154062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a902ce09-6068-4964-9502-3518c9690fc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:17.154062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a902ce09-6068-4964-9502-3518c9690fc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:17.154533 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a902ce09-6068-4964-9502-3518c9690fc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:17.173172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-a902ce09-6068-4964-9502-3518c9690fc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lazy-loading 'flavor' on Instance uuid 2707cec6-5b8c-41b5-8ae3-1b4c4de591af {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 13:55:17.522355 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a902ce09-6068-4964-9502-3518c9690fc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "POST /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af/action" status: 200 len: 32 microversion: 2.1 time: 0.390892 Sep 08 13:55:17.527676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 114/233] 10.222.0.25 () {60 vars in 1278 bytes} [Thu Sep 8 13:55:17 2022] POST /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af/action => generated 32 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 08 13:55:17.535483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-316c14bc-4093-42d3-9ddd-5d38d4aef687 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:17.536638 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-316c14bc-4093-42d3-9ddd-5d38d4aef687 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:17.543394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-316c14bc-4093-42d3-9ddd-5d38d4aef687 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:17.544176 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-316c14bc-4093-42d3-9ddd-5d38d4aef687 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:17.544808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-316c14bc-4093-42d3-9ddd-5d38d4aef687 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:17.545744 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-316c14bc-4093-42d3-9ddd-5d38d4aef687 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:17.546295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-316c14bc-4093-42d3-9ddd-5d38d4aef687 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:17.546786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-316c14bc-4093-42d3-9ddd-5d38d4aef687 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:17.684168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-316c14bc-4093-42d3-9ddd-5d38d4aef687 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:17.692812 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-316c14bc-4093-42d3-9ddd-5d38d4aef687 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.162747 Sep 08 13:55:17.693375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 120/234] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:17 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:17.958057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-501343d5-6793-4451-8c13-935d55672bf5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:17.958855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-501343d5-6793-4451-8c13-935d55672bf5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:17.965643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-501343d5-6793-4451-8c13-935d55672bf5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:17.966171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-501343d5-6793-4451-8c13-935d55672bf5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:17.966598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-501343d5-6793-4451-8c13-935d55672bf5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:17.967521 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-501343d5-6793-4451-8c13-935d55672bf5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:17.968134 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-501343d5-6793-4451-8c13-935d55672bf5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:17.968748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-501343d5-6793-4451-8c13-935d55672bf5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:18.134982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-501343d5-6793-4451-8c13-935d55672bf5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] HTTP exception thrown: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-501343d5-6793-4451-8c13-935d55672bf5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Request method failure captured: Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 HTTP/1.1 Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPolVQrFmzJ1tP4tqIdAFktUoN_I-dkkz77om_IARYStbSsI7lAhhXf3o6yPalI8qFGff3S2naq5_Obm3YPQDrUDoB0Jyv9ACMCFtuErncM6uQV9c4D1tl8ur2B4mC7EcT7Io5CR0SrYMzlnBJK25Kv4EQnhwNWkmpNzX8ls27UnNU Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 5775050045e749b1bdb26dddcce9a8ff Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-DeleteServersTestJSON-2088877436 Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 13:55:18.136894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 13:55:18.138689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_5775050045e749b1bdb26dddcce9a8ff"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5775050045e749b1bdb26dddcce9a8ff"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5775050045e749b1bdb26dddcce9a8ff"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/5775050045e749b1bdb26dddcce9a8ff"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:55:18.138689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-DeleteServersTestJSON-2088877436 Sep 08 13:55:18.138689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 5775050045e749b1bdb26dddcce9a8ff Sep 08 13:55:18.138689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-DeleteServersTestJSON-2088877436 Sep 08 13:55:18.138689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-DeleteServersTestJSON-2088877436-project Sep 08 13:55:18.138689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 13:55:18.138689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 13:55:18.138689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 8294dcc1178045abbcf7a497564eb122 Sep 08 13:55:18.138689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-DeleteServersTestJSON-2088877436-project Sep 08 13:55:18.138689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 13:55:18.138689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:18.138689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': '358e292a-9e96-4380-8ea5-3ed00372b4c3'} Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:55:18.139878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:18.140963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:55:18.143228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-501343d5-6793-4451-8c13-935d55672bf5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Returning 404 to user: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:55:18.143228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-501343d5-6793-4451-8c13-935d55672bf5 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "DELETE /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 404 len: 111 microversion: 2.1 time: 0.184932 Sep 08 13:55:18.143228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 115/235] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 13:55:17 2022] DELETE /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 111 bytes in 186 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 13:55:18.147593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-930b4f40-2695-416b-8d0e-4ebac08eb741 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:18.151135 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-930b4f40-2695-416b-8d0e-4ebac08eb741 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] [instance: 358e292a-9e96-4380-8ea5-3ed00372b4c3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:18.154880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-930b4f40-2695-416b-8d0e-4ebac08eb741 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:18.155369 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-930b4f40-2695-416b-8d0e-4ebac08eb741 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:18.155798 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-930b4f40-2695-416b-8d0e-4ebac08eb741 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:18.156494 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-930b4f40-2695-416b-8d0e-4ebac08eb741 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:18.156941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-930b4f40-2695-416b-8d0e-4ebac08eb741 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:18.157638 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-930b4f40-2695-416b-8d0e-4ebac08eb741 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:18.172056 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-930b4f40-2695-416b-8d0e-4ebac08eb741 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] HTTP exception thrown: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-930b4f40-2695-416b-8d0e-4ebac08eb741 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Request method failure captured: Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 HTTP/1.1 Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPolVQrFmzJ1tP4tqIdAFktUoN_I-dkkz77om_IARYStbSsI7lAhhXf3o6yPalI8qFGff3S2naq5_Obm3YPQDrUDoB0Jyv9ACMCFtuErncM6uQV9c4D1tl8ur2B4mC7EcT7Io5CR0SrYMzlnBJK25Kv4EQnhwNWkmpNzX8ls27UnNU Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 5775050045e749b1bdb26dddcce9a8ff Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-DeleteServersTestJSON-2088877436 Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 13:55:18.174761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 13:55:18.178461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_5775050045e749b1bdb26dddcce9a8ff"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5775050045e749b1bdb26dddcce9a8ff"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5775050045e749b1bdb26dddcce9a8ff"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/5775050045e749b1bdb26dddcce9a8ff"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:55:18.178461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-DeleteServersTestJSON-2088877436 Sep 08 13:55:18.178461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 5775050045e749b1bdb26dddcce9a8ff Sep 08 13:55:18.178461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-DeleteServersTestJSON-2088877436 Sep 08 13:55:18.178461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-DeleteServersTestJSON-2088877436-project Sep 08 13:55:18.178461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 13:55:18.178461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 13:55:18.178461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 8294dcc1178045abbcf7a497564eb122 Sep 08 13:55:18.178461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-DeleteServersTestJSON-2088877436-project Sep 08 13:55:18.178461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 13:55:18.178461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:18.178461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '358e292a-9e96-4380-8ea5-3ed00372b4c3'} Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:55:18.179229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:55:18.180325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:55:18.180325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 13:55:18.180325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 13:55:18.180325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 13:55:18.180325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 13:55:18.180325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 13:55:18.180325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 13:55:18.180325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:18.180325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:55:18.180325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:55:18.180325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:55:18.180325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:55:18.180325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:55:18.180325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:55:18.180325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:55:18.180325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:55:18.180325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. Sep 08 13:55:18.180325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:55:18.181303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-930b4f40-2695-416b-8d0e-4ebac08eb741 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] Returning 404 to user: Instance 358e292a-9e96-4380-8ea5-3ed00372b4c3 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:55:18.181303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-930b4f40-2695-416b-8d0e-4ebac08eb741 tempest-DeleteServersTestJSON-2088877436 tempest-DeleteServersTestJSON-2088877436-project] 10.222.0.25 "GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3" status: 404 len: 111 microversion: 2.1 time: 0.033226 Sep 08 13:55:18.182669 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 121/236] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:18 2022] GET /compute/v2.1/servers/358e292a-9e96-4380-8ea5-3ed00372b4c3 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 13:55:18.720047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3bff99f3-afdd-4817-bdc8-396d429f7760 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:18.720971 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3bff99f3-afdd-4817-bdc8-396d429f7760 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:18.731970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3bff99f3-afdd-4817-bdc8-396d429f7760 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:18.733053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3bff99f3-afdd-4817-bdc8-396d429f7760 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:18.733979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3bff99f3-afdd-4817-bdc8-396d429f7760 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:18.734850 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3bff99f3-afdd-4817-bdc8-396d429f7760 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:18.735559 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3bff99f3-afdd-4817-bdc8-396d429f7760 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:18.736213 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3bff99f3-afdd-4817-bdc8-396d429f7760 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:19.206494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3bff99f3-afdd-4817-bdc8-396d429f7760 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:19.211398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3bff99f3-afdd-4817-bdc8-396d429f7760 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.498002 Sep 08 13:55:19.212661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 116/237] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:18 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:55:20.244125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-582a3ff1-c3be-4bf6-b7d3-37f6c5dd5f47 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:20.244125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-582a3ff1-c3be-4bf6-b7d3-37f6c5dd5f47 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:20.248385 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-582a3ff1-c3be-4bf6-b7d3-37f6c5dd5f47 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:20.249134 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-582a3ff1-c3be-4bf6-b7d3-37f6c5dd5f47 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:20.249569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-582a3ff1-c3be-4bf6-b7d3-37f6c5dd5f47 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:20.250311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-582a3ff1-c3be-4bf6-b7d3-37f6c5dd5f47 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:20.250945 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-582a3ff1-c3be-4bf6-b7d3-37f6c5dd5f47 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:20.251375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-582a3ff1-c3be-4bf6-b7d3-37f6c5dd5f47 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:20.444450 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-582a3ff1-c3be-4bf6-b7d3-37f6c5dd5f47 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:20.449533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-582a3ff1-c3be-4bf6-b7d3-37f6c5dd5f47 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.213036 Sep 08 13:55:20.450471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 122/238] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:20 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:21.477771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c0b64442-0da2-421d-8454-a3e33da115e3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:21.477771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c0b64442-0da2-421d-8454-a3e33da115e3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:21.480477 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0b64442-0da2-421d-8454-a3e33da115e3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:21.480973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0b64442-0da2-421d-8454-a3e33da115e3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:21.481416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0b64442-0da2-421d-8454-a3e33da115e3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:21.482228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0b64442-0da2-421d-8454-a3e33da115e3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:21.482779 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0b64442-0da2-421d-8454-a3e33da115e3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:21.483142 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0b64442-0da2-421d-8454-a3e33da115e3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:21.638782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c0b64442-0da2-421d-8454-a3e33da115e3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:21.647969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c0b64442-0da2-421d-8454-a3e33da115e3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.178813 Sep 08 13:55:21.648542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 117/239] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:21 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:22.672965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-08236720-254e-4507-8be6-27c20392fecf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:22.674904 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-08236720-254e-4507-8be6-27c20392fecf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:22.682672 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08236720-254e-4507-8be6-27c20392fecf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:22.683153 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08236720-254e-4507-8be6-27c20392fecf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:22.683577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08236720-254e-4507-8be6-27c20392fecf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:22.684266 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08236720-254e-4507-8be6-27c20392fecf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:22.684836 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08236720-254e-4507-8be6-27c20392fecf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:22.685199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08236720-254e-4507-8be6-27c20392fecf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:22.849548 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-08236720-254e-4507-8be6-27c20392fecf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:22.858200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-08236720-254e-4507-8be6-27c20392fecf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.190847 Sep 08 13:55:22.858611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 123/240] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:22 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:23.886526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-aaba4bd7-c7b7-46eb-955b-f0e16bb9609d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:23.887123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-aaba4bd7-c7b7-46eb-955b-f0e16bb9609d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:23.893811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aaba4bd7-c7b7-46eb-955b-f0e16bb9609d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:23.894481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aaba4bd7-c7b7-46eb-955b-f0e16bb9609d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:23.897200 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aaba4bd7-c7b7-46eb-955b-f0e16bb9609d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:23.898029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aaba4bd7-c7b7-46eb-955b-f0e16bb9609d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:23.898580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aaba4bd7-c7b7-46eb-955b-f0e16bb9609d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:23.899327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aaba4bd7-c7b7-46eb-955b-f0e16bb9609d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:24.035039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-aaba4bd7-c7b7-46eb-955b-f0e16bb9609d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:24.049365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-aaba4bd7-c7b7-46eb-955b-f0e16bb9609d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.167401 Sep 08 13:55:24.052567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 118/241] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:23 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:25.100703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bfbda7f3-9ba0-41ff-9b46-20c5507e83f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:25.100703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bfbda7f3-9ba0-41ff-9b46-20c5507e83f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:25.109026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bfbda7f3-9ba0-41ff-9b46-20c5507e83f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:25.109855 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bfbda7f3-9ba0-41ff-9b46-20c5507e83f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:25.110559 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bfbda7f3-9ba0-41ff-9b46-20c5507e83f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:25.111366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bfbda7f3-9ba0-41ff-9b46-20c5507e83f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:25.112154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bfbda7f3-9ba0-41ff-9b46-20c5507e83f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:25.112677 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bfbda7f3-9ba0-41ff-9b46-20c5507e83f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:25.290092 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-bfbda7f3-9ba0-41ff-9b46-20c5507e83f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:25.295026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bfbda7f3-9ba0-41ff-9b46-20c5507e83f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.199986 Sep 08 13:55:25.295685 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 124/242] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:25 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:26.321087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-95ca1eae-1146-4f4d-8616-f1e15efb3c6b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:26.322430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-95ca1eae-1146-4f4d-8616-f1e15efb3c6b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:26.329335 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-95ca1eae-1146-4f4d-8616-f1e15efb3c6b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:26.329833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-95ca1eae-1146-4f4d-8616-f1e15efb3c6b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:26.330541 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-95ca1eae-1146-4f4d-8616-f1e15efb3c6b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:26.331440 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-95ca1eae-1146-4f4d-8616-f1e15efb3c6b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:26.332618 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-95ca1eae-1146-4f4d-8616-f1e15efb3c6b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:26.333332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-95ca1eae-1146-4f4d-8616-f1e15efb3c6b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:26.591586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-95ca1eae-1146-4f4d-8616-f1e15efb3c6b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:26.600060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-95ca1eae-1146-4f4d-8616-f1e15efb3c6b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.283039 Sep 08 13:55:26.600623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 119/243] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:26 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:27.624319 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0c522012-3319-4acc-819c-59e05a7b017b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:27.624932 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0c522012-3319-4acc-819c-59e05a7b017b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:27.631075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0c522012-3319-4acc-819c-59e05a7b017b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:27.631557 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0c522012-3319-4acc-819c-59e05a7b017b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:27.631990 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0c522012-3319-4acc-819c-59e05a7b017b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:27.632687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0c522012-3319-4acc-819c-59e05a7b017b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:27.633170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0c522012-3319-4acc-819c-59e05a7b017b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:27.639201 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0c522012-3319-4acc-819c-59e05a7b017b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:27.787361 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0c522012-3319-4acc-819c-59e05a7b017b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:27.795756 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0c522012-3319-4acc-819c-59e05a7b017b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.176198 Sep 08 13:55:27.796326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 125/244] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:27 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:28.818899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-db229a65-2012-4018-951e-5f5821e460ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:28.819424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-db229a65-2012-4018-951e-5f5821e460ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:28.826312 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db229a65-2012-4018-951e-5f5821e460ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:28.826831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db229a65-2012-4018-951e-5f5821e460ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:28.827272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db229a65-2012-4018-951e-5f5821e460ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:28.827972 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db229a65-2012-4018-951e-5f5821e460ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:28.828431 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db229a65-2012-4018-951e-5f5821e460ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:28.828862 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db229a65-2012-4018-951e-5f5821e460ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:28.973258 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-db229a65-2012-4018-951e-5f5821e460ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:28.981814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-db229a65-2012-4018-951e-5f5821e460ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.166717 Sep 08 13:55:28.982439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 120/245] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:28 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:30.007674 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b1118551-9340-411a-99fb-72f643188e54 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:30.008872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b1118551-9340-411a-99fb-72f643188e54 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:30.017862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1118551-9340-411a-99fb-72f643188e54 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:30.018621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1118551-9340-411a-99fb-72f643188e54 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:30.019058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1118551-9340-411a-99fb-72f643188e54 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:30.019973 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1118551-9340-411a-99fb-72f643188e54 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:30.020436 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1118551-9340-411a-99fb-72f643188e54 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:30.020886 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1118551-9340-411a-99fb-72f643188e54 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:30.186714 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b1118551-9340-411a-99fb-72f643188e54 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:30.195481 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b1118551-9340-411a-99fb-72f643188e54 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.192914 Sep 08 13:55:30.206164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 126/246] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:30 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:31.226800 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c634c0d1-af3a-485a-9c5c-4ed112c016ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:31.231398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c634c0d1-af3a-485a-9c5c-4ed112c016ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:31.235093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c634c0d1-af3a-485a-9c5c-4ed112c016ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:31.235663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c634c0d1-af3a-485a-9c5c-4ed112c016ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:31.236281 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c634c0d1-af3a-485a-9c5c-4ed112c016ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:31.236992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c634c0d1-af3a-485a-9c5c-4ed112c016ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:31.237454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c634c0d1-af3a-485a-9c5c-4ed112c016ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:31.237897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c634c0d1-af3a-485a-9c5c-4ed112c016ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:31.400816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c634c0d1-af3a-485a-9c5c-4ed112c016ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:31.408612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c634c0d1-af3a-485a-9c5c-4ed112c016ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.185723 Sep 08 13:55:31.409306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 121/247] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:31 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:32.433348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bd8ccf70-bbec-4d33-b23f-cfb0af383ec9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:32.433967 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bd8ccf70-bbec-4d33-b23f-cfb0af383ec9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:32.440483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd8ccf70-bbec-4d33-b23f-cfb0af383ec9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:32.440963 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd8ccf70-bbec-4d33-b23f-cfb0af383ec9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:32.441425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd8ccf70-bbec-4d33-b23f-cfb0af383ec9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:32.442668 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd8ccf70-bbec-4d33-b23f-cfb0af383ec9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:32.443115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd8ccf70-bbec-4d33-b23f-cfb0af383ec9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:32.443536 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd8ccf70-bbec-4d33-b23f-cfb0af383ec9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:32.596145 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-bd8ccf70-bbec-4d33-b23f-cfb0af383ec9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:32.608522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bd8ccf70-bbec-4d33-b23f-cfb0af383ec9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.178771 Sep 08 13:55:32.609488 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 127/248] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:32 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:33.639013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-11a74464-53f2-4053-80f1-ebe1273374a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:33.641298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-11a74464-53f2-4053-80f1-ebe1273374a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:33.649213 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-11a74464-53f2-4053-80f1-ebe1273374a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:33.649818 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-11a74464-53f2-4053-80f1-ebe1273374a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:33.650286 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-11a74464-53f2-4053-80f1-ebe1273374a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:33.651292 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-11a74464-53f2-4053-80f1-ebe1273374a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:33.655282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-11a74464-53f2-4053-80f1-ebe1273374a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:33.655733 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-11a74464-53f2-4053-80f1-ebe1273374a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:33.802122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-11a74464-53f2-4053-80f1-ebe1273374a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:33.815127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-11a74464-53f2-4053-80f1-ebe1273374a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.181699 Sep 08 13:55:33.816204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 122/249] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:33 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:34.839714 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4d7547e9-b484-4ee8-88ac-7161cf38c3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:34.840327 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4d7547e9-b484-4ee8-88ac-7161cf38c3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:34.843880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=34 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:34.847246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d7547e9-b484-4ee8-88ac-7161cf38c3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:34.847738 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d7547e9-b484-4ee8-88ac-7161cf38c3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:34.848685 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d7547e9-b484-4ee8-88ac-7161cf38c3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:34.849384 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d7547e9-b484-4ee8-88ac-7161cf38c3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:34.849834 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d7547e9-b484-4ee8-88ac-7161cf38c3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:34.850281 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d7547e9-b484-4ee8-88ac-7161cf38c3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:35.016105 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4d7547e9-b484-4ee8-88ac-7161cf38c3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:35.018889 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=81 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:35.026551 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4d7547e9-b484-4ee8-88ac-7161cf38c3e1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.191006 Sep 08 13:55:35.027127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 128/250] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:34 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:36.056016 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-190d1f08-fd75-4b82-bf0c-5d0b001fd2f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:36.056607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-190d1f08-fd75-4b82-bf0c-5d0b001fd2f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:36.065373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-190d1f08-fd75-4b82-bf0c-5d0b001fd2f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:36.065373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-190d1f08-fd75-4b82-bf0c-5d0b001fd2f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:36.065373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-190d1f08-fd75-4b82-bf0c-5d0b001fd2f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:36.065373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-190d1f08-fd75-4b82-bf0c-5d0b001fd2f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:36.065936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-190d1f08-fd75-4b82-bf0c-5d0b001fd2f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:36.065936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-190d1f08-fd75-4b82-bf0c-5d0b001fd2f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:36.218256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-190d1f08-fd75-4b82-bf0c-5d0b001fd2f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:36.227523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-190d1f08-fd75-4b82-bf0c-5d0b001fd2f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.176271 Sep 08 13:55:36.228090 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 123/251] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:36 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:37.258393 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-79820824-7fce-4852-a419-f880c26142f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:37.258393 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-79820824-7fce-4852-a419-f880c26142f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:37.259920 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79820824-7fce-4852-a419-f880c26142f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:37.260558 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79820824-7fce-4852-a419-f880c26142f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:37.261155 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79820824-7fce-4852-a419-f880c26142f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:37.261918 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79820824-7fce-4852-a419-f880c26142f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:37.262458 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79820824-7fce-4852-a419-f880c26142f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:37.262951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79820824-7fce-4852-a419-f880c26142f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:37.395939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-79820824-7fce-4852-a419-f880c26142f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:37.403614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-79820824-7fce-4852-a419-f880c26142f7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.156070 Sep 08 13:55:37.404177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 129/252] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:37 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:38.427389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-36d3fb32-0822-4344-a1f5-1445a804d11a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:38.428557 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-36d3fb32-0822-4344-a1f5-1445a804d11a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:38.435829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-36d3fb32-0822-4344-a1f5-1445a804d11a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:38.436327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-36d3fb32-0822-4344-a1f5-1445a804d11a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:38.436761 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-36d3fb32-0822-4344-a1f5-1445a804d11a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:38.437458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-36d3fb32-0822-4344-a1f5-1445a804d11a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:38.437910 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-36d3fb32-0822-4344-a1f5-1445a804d11a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:38.438361 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-36d3fb32-0822-4344-a1f5-1445a804d11a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:38.604607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-36d3fb32-0822-4344-a1f5-1445a804d11a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:38.613381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-36d3fb32-0822-4344-a1f5-1445a804d11a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.190661 Sep 08 13:55:38.619410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 124/253] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:38 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:39.637934 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1f12b8ef-c9b3-444f-95b7-f57224c9f596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:39.638968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1f12b8ef-c9b3-444f-95b7-f57224c9f596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:39.646343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f12b8ef-c9b3-444f-95b7-f57224c9f596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:39.646827 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f12b8ef-c9b3-444f-95b7-f57224c9f596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:39.647258 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f12b8ef-c9b3-444f-95b7-f57224c9f596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:39.647961 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f12b8ef-c9b3-444f-95b7-f57224c9f596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:39.648442 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f12b8ef-c9b3-444f-95b7-f57224c9f596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:39.648862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f12b8ef-c9b3-444f-95b7-f57224c9f596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:39.800189 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1f12b8ef-c9b3-444f-95b7-f57224c9f596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:39.808268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1f12b8ef-c9b3-444f-95b7-f57224c9f596 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.175879 Sep 08 13:55:39.808873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 130/254] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:39 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:40.832060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cfd3d3dd-a890-4dcb-b7ff-0ecd535bddaf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:40.833399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cfd3d3dd-a890-4dcb-b7ff-0ecd535bddaf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:40.839829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cfd3d3dd-a890-4dcb-b7ff-0ecd535bddaf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:40.840318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cfd3d3dd-a890-4dcb-b7ff-0ecd535bddaf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:40.840750 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cfd3d3dd-a890-4dcb-b7ff-0ecd535bddaf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:40.841435 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cfd3d3dd-a890-4dcb-b7ff-0ecd535bddaf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:40.841880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cfd3d3dd-a890-4dcb-b7ff-0ecd535bddaf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:40.842331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cfd3d3dd-a890-4dcb-b7ff-0ecd535bddaf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:40.847612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=88,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:40.987218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cfd3d3dd-a890-4dcb-b7ff-0ecd535bddaf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:40.996110 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cfd3d3dd-a890-4dcb-b7ff-0ecd535bddaf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.167969 Sep 08 13:55:40.998480 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 125/255] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:40 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:42.021642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ea93c462-45c6-4110-b9a7-2630b49ff59f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:42.022285 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ea93c462-45c6-4110-b9a7-2630b49ff59f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:42.028384 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ea93c462-45c6-4110-b9a7-2630b49ff59f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:42.028864 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ea93c462-45c6-4110-b9a7-2630b49ff59f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:42.029297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ea93c462-45c6-4110-b9a7-2630b49ff59f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:42.029987 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ea93c462-45c6-4110-b9a7-2630b49ff59f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:42.030482 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ea93c462-45c6-4110-b9a7-2630b49ff59f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:42.030906 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ea93c462-45c6-4110-b9a7-2630b49ff59f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:42.178643 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ea93c462-45c6-4110-b9a7-2630b49ff59f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:42.186695 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ea93c462-45c6-4110-b9a7-2630b49ff59f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1502 microversion: 2.1 time: 0.169701 Sep 08 13:55:42.189529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 131/256] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:42 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1502 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:42.212965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-eac4831c-46f7-4850-bcfd-32342163b7a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "8b885a57-13f9-4e8f-9fd1-e1fffd5910cf"}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:55:42.215308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-eac4831c-46f7-4850-bcfd-32342163b7a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:42.222414 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eac4831c-46f7-4850-bcfd-32342163b7a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:42.226650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eac4831c-46f7-4850-bcfd-32342163b7a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:42.226650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eac4831c-46f7-4850-bcfd-32342163b7a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:42.226650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eac4831c-46f7-4850-bcfd-32342163b7a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:42.226650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eac4831c-46f7-4850-bcfd-32342163b7a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:42.227223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eac4831c-46f7-4850-bcfd-32342163b7a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:42.247537 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-eac4831c-46f7-4850-bcfd-32342163b7a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] HTTP exception thrown: Cannot 'attach_volume' instance 2707cec6-5b8c-41b5-8ae3-1b4c4de591af while it is in vm_state rescued Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-eac4831c-46f7-4850-bcfd-32342163b7a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Request method failure captured: Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: POST /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af/os-volume_attachments HTTP/1.1 Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 74 Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPpGZzMisIeN0rinm76oerWeI1e2n1ZCDvu1_QxGMaqly5J66d47JWYR-nVcthn5vbK6XFUxRx7ULj4gJs_JU089dQK1DqQD0zha4-7jxb_89w0QMqZgtACCW34zdnpCyIfB12dgyAWUWv38AgePo6JrTClIfdFoSLdR5OVWnj2kmY Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 1346d4105dce4c6e928fcb11cde56854 Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-188566618 Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 13:55:42.249626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 13:55:42.251229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:55:42.251229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-ServerRescueNegativeTestJSON-188566618 Sep 08 13:55:42.251229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 1346d4105dce4c6e928fcb11cde56854 Sep 08 13:55:42.251229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-188566618 Sep 08 13:55:42.251229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-ServerRescueNegativeTestJSON-188566618-project Sep 08 13:55:42.251229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 13:55:42.251229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 13:55:42.251229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 792808dcbf894aa98c671cf43ac21b8b Sep 08 13:55:42.251229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-ServerRescueNegativeTestJSON-188566618-project Sep 08 13:55:42.251229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Sep 08 13:55:42.251229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {"volumeAttachment": {"volumeId": "8b885a57-13f9-4e8f-9fd1-e1fffd5910cf"}} Sep 08 13:55:42.251229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Cannot 'attach_volume' instance 2707cec6-5b8c-41b5-8ae3-1b4c4de591af while it is in vm_state rescued Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'server_id': '2707cec6-5b8c-41b5-8ae3-1b4c4de591af', 'body': {'volumeAttachment': {'volumeId': '8b885a57-13f9-4e8f-9fd1-e1fffd5910cf'}}} Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 2707cec6-5b8c-41b5-8ae3-1b4c4de591af in vm_state rescued. Cannot attach_volume while the instance is in this state. Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:55:42.252109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 2707cec6-5b8c-41b5-8ae3-1b4c4de591af while it is in vm_state rescued Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:55:42.253520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:55:42.255517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:55:42.255517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:55:42.255517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 2707cec6-5b8c-41b5-8ae3-1b4c4de591af while it is in vm_state rescued Sep 08 13:55:42.255517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:55:42.255517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-eac4831c-46f7-4850-bcfd-32342163b7a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Returning 409 to user: Cannot 'attach_volume' instance 2707cec6-5b8c-41b5-8ae3-1b4c4de591af while it is in vm_state rescued {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:55:42.255517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-eac4831c-46f7-4850-bcfd-32342163b7a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "POST /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.042195 Sep 08 13:55:42.255517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 126/257] 10.222.0.25 () {60 vars in 1323 bytes} [Thu Sep 8 13:55:42 2022] POST /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af/os-volume_attachments => generated 152 bytes in 44 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 08 13:55:42.260932 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ca15386c-ff09-4258-88ec-ab7d5b31ebb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:55:42.261498 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ca15386c-ff09-4258-88ec-ab7d5b31ebb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:42.267432 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca15386c-ff09-4258-88ec-ab7d5b31ebb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:42.267915 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca15386c-ff09-4258-88ec-ab7d5b31ebb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:42.268346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca15386c-ff09-4258-88ec-ab7d5b31ebb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:42.269291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca15386c-ff09-4258-88ec-ab7d5b31ebb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:42.269748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca15386c-ff09-4258-88ec-ab7d5b31ebb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:42.270863 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca15386c-ff09-4258-88ec-ab7d5b31ebb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:42.495785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ca15386c-ff09-4258-88ec-ab7d5b31ebb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "POST /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af/action" status: 202 len: 0 microversion: 2.1 time: 0.238498 Sep 08 13:55:42.496425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 132/258] 10.222.0.25 () {60 vars in 1278 bytes} [Thu Sep 8 13:55:42 2022] POST /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af/action => generated 0 bytes in 240 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 13:55:42.505445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-61eb8b98-915f-47d4-aac1-96c876a4e2cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:42.506248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-61eb8b98-915f-47d4-aac1-96c876a4e2cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:42.512340 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61eb8b98-915f-47d4-aac1-96c876a4e2cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:42.512829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61eb8b98-915f-47d4-aac1-96c876a4e2cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:42.513269 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61eb8b98-915f-47d4-aac1-96c876a4e2cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:42.513968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61eb8b98-915f-47d4-aac1-96c876a4e2cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:42.514461 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61eb8b98-915f-47d4-aac1-96c876a4e2cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:42.514887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61eb8b98-915f-47d4-aac1-96c876a4e2cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:42.647065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-61eb8b98-915f-47d4-aac1-96c876a4e2cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:42.654841 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-61eb8b98-915f-47d4-aac1-96c876a4e2cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1510 microversion: 2.1 time: 0.153800 Sep 08 13:55:42.655447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 127/259] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:42 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1510 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:43.678830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5f784918-62c2-4d12-92c5-75828dfaa4a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:43.679556 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5f784918-62c2-4d12-92c5-75828dfaa4a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:43.685889 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5f784918-62c2-4d12-92c5-75828dfaa4a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:43.686507 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5f784918-62c2-4d12-92c5-75828dfaa4a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:43.686942 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5f784918-62c2-4d12-92c5-75828dfaa4a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:43.687634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5f784918-62c2-4d12-92c5-75828dfaa4a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:43.688086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5f784918-62c2-4d12-92c5-75828dfaa4a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:43.688505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5f784918-62c2-4d12-92c5-75828dfaa4a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:43.831054 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5f784918-62c2-4d12-92c5-75828dfaa4a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:43.843071 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5f784918-62c2-4d12-92c5-75828dfaa4a7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1510 microversion: 2.1 time: 0.168488 Sep 08 13:55:43.844325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 133/260] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:43 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1510 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:44.867917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-66e0f6fb-02ca-4736-b246-e90d54d53e5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:44.868518 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-66e0f6fb-02ca-4736-b246-e90d54d53e5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:44.875461 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66e0f6fb-02ca-4736-b246-e90d54d53e5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:44.875954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66e0f6fb-02ca-4736-b246-e90d54d53e5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:44.876389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66e0f6fb-02ca-4736-b246-e90d54d53e5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:44.877091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66e0f6fb-02ca-4736-b246-e90d54d53e5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:44.877545 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66e0f6fb-02ca-4736-b246-e90d54d53e5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:44.877970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66e0f6fb-02ca-4736-b246-e90d54d53e5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:45.028494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-66e0f6fb-02ca-4736-b246-e90d54d53e5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:45.037283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-66e0f6fb-02ca-4736-b246-e90d54d53e5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1510 microversion: 2.1 time: 0.173308 Sep 08 13:55:45.037887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 128/261] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:44 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1510 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:46.060985 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ca0c7ca5-4fb5-4fc1-820c-9b1a168ee08f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:46.061579 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ca0c7ca5-4fb5-4fc1-820c-9b1a168ee08f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:46.068190 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca0c7ca5-4fb5-4fc1-820c-9b1a168ee08f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:46.068600 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca0c7ca5-4fb5-4fc1-820c-9b1a168ee08f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:46.069023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca0c7ca5-4fb5-4fc1-820c-9b1a168ee08f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:46.069709 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca0c7ca5-4fb5-4fc1-820c-9b1a168ee08f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:46.070174 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca0c7ca5-4fb5-4fc1-820c-9b1a168ee08f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:46.070593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca0c7ca5-4fb5-4fc1-820c-9b1a168ee08f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:46.206099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ca0c7ca5-4fb5-4fc1-820c-9b1a168ee08f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:46.214938 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ca0c7ca5-4fb5-4fc1-820c-9b1a168ee08f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1510 microversion: 2.1 time: 0.157641 Sep 08 13:55:46.217260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 134/262] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:46 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1510 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:47.242211 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f54acfe2-c3ee-479f-a027-fac6ffdc8905 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:47.242211 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f54acfe2-c3ee-479f-a027-fac6ffdc8905 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:47.248315 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f54acfe2-c3ee-479f-a027-fac6ffdc8905 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:47.249161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f54acfe2-c3ee-479f-a027-fac6ffdc8905 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:47.249953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f54acfe2-c3ee-479f-a027-fac6ffdc8905 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:47.250798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f54acfe2-c3ee-479f-a027-fac6ffdc8905 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:47.251254 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f54acfe2-c3ee-479f-a027-fac6ffdc8905 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:47.251679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f54acfe2-c3ee-479f-a027-fac6ffdc8905 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:47.420219 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f54acfe2-c3ee-479f-a027-fac6ffdc8905 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:47.427961 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f54acfe2-c3ee-479f-a027-fac6ffdc8905 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1510 microversion: 2.1 time: 0.193110 Sep 08 13:55:47.428528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 129/263] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:47 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1510 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:48.450890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b696c23d-fb8e-4306-8b7c-f96fce934d56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:48.451590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b696c23d-fb8e-4306-8b7c-f96fce934d56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:48.457629 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b696c23d-fb8e-4306-8b7c-f96fce934d56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:48.458114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b696c23d-fb8e-4306-8b7c-f96fce934d56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:48.458600 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b696c23d-fb8e-4306-8b7c-f96fce934d56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:48.459295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b696c23d-fb8e-4306-8b7c-f96fce934d56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:48.459744 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b696c23d-fb8e-4306-8b7c-f96fce934d56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:48.460167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b696c23d-fb8e-4306-8b7c-f96fce934d56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:48.619173 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b696c23d-fb8e-4306-8b7c-f96fce934d56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:48.638954 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b696c23d-fb8e-4306-8b7c-f96fce934d56 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1510 microversion: 2.1 time: 0.191200 Sep 08 13:55:48.639754 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 135/264] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:48 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1510 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:49.664200 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8b4fc3dd-4449-4a2e-9645-4cf43bf28cf0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:49.664888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8b4fc3dd-4449-4a2e-9645-4cf43bf28cf0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:49.668670 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=32 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:49.674887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b4fc3dd-4449-4a2e-9645-4cf43bf28cf0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:49.678463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b4fc3dd-4449-4a2e-9645-4cf43bf28cf0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:49.678986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b4fc3dd-4449-4a2e-9645-4cf43bf28cf0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:49.680373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b4fc3dd-4449-4a2e-9645-4cf43bf28cf0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:49.681185 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b4fc3dd-4449-4a2e-9645-4cf43bf28cf0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:49.681980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b4fc3dd-4449-4a2e-9645-4cf43bf28cf0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:49.872162 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8b4fc3dd-4449-4a2e-9645-4cf43bf28cf0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:49.880423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8b4fc3dd-4449-4a2e-9645-4cf43bf28cf0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1510 microversion: 2.1 time: 0.220577 Sep 08 13:55:49.881005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 130/265] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:49 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1510 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:50.905356 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5955c9b8-6549-451e-b20b-85bb1221f17d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:50.905963 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5955c9b8-6549-451e-b20b-85bb1221f17d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:50.912122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5955c9b8-6549-451e-b20b-85bb1221f17d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:50.912611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5955c9b8-6549-451e-b20b-85bb1221f17d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:50.913038 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5955c9b8-6549-451e-b20b-85bb1221f17d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:50.913731 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5955c9b8-6549-451e-b20b-85bb1221f17d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:50.914210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5955c9b8-6549-451e-b20b-85bb1221f17d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:50.914628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5955c9b8-6549-451e-b20b-85bb1221f17d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:51.059106 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5955c9b8-6549-451e-b20b-85bb1221f17d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:51.067789 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5955c9b8-6549-451e-b20b-85bb1221f17d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1510 microversion: 2.1 time: 0.165860 Sep 08 13:55:51.068435 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 136/266] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:50 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1510 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:52.090215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9b700ca1-168c-40eb-bfd1-18d8f5600e75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:52.090828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9b700ca1-168c-40eb-bfd1-18d8f5600e75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:52.097963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9b700ca1-168c-40eb-bfd1-18d8f5600e75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:52.098575 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9b700ca1-168c-40eb-bfd1-18d8f5600e75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:52.099257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9b700ca1-168c-40eb-bfd1-18d8f5600e75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:52.100171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9b700ca1-168c-40eb-bfd1-18d8f5600e75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:52.100694 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9b700ca1-168c-40eb-bfd1-18d8f5600e75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:52.103977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9b700ca1-168c-40eb-bfd1-18d8f5600e75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:52.264845 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9b700ca1-168c-40eb-bfd1-18d8f5600e75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:52.273407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9b700ca1-168c-40eb-bfd1-18d8f5600e75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1510 microversion: 2.1 time: 0.187144 Sep 08 13:55:52.276104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 131/267] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:52 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1510 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:53.300802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-39aaa5c5-ef79-480a-aa7e-ba6596c867f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:53.301434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-39aaa5c5-ef79-480a-aa7e-ba6596c867f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:53.307679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39aaa5c5-ef79-480a-aa7e-ba6596c867f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:53.308171 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39aaa5c5-ef79-480a-aa7e-ba6596c867f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:53.308604 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39aaa5c5-ef79-480a-aa7e-ba6596c867f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:53.309298 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39aaa5c5-ef79-480a-aa7e-ba6596c867f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:53.309755 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39aaa5c5-ef79-480a-aa7e-ba6596c867f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:53.310246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39aaa5c5-ef79-480a-aa7e-ba6596c867f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:53.466377 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-39aaa5c5-ef79-480a-aa7e-ba6596c867f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:53.475576 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-39aaa5c5-ef79-480a-aa7e-ba6596c867f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1510 microversion: 2.1 time: 0.178461 Sep 08 13:55:53.476191 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 137/268] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:53 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1510 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:54.501801 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-376876dc-3c5f-499b-8bfa-714c4535b861 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:54.502571 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-376876dc-3c5f-499b-8bfa-714c4535b861 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:54.511551 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-376876dc-3c5f-499b-8bfa-714c4535b861 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:54.512308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-376876dc-3c5f-499b-8bfa-714c4535b861 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:54.512755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-376876dc-3c5f-499b-8bfa-714c4535b861 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:54.513666 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-376876dc-3c5f-499b-8bfa-714c4535b861 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:54.514130 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-376876dc-3c5f-499b-8bfa-714c4535b861 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:54.514594 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-376876dc-3c5f-499b-8bfa-714c4535b861 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:54.691580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-376876dc-3c5f-499b-8bfa-714c4535b861 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:55:54.699465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-376876dc-3c5f-499b-8bfa-714c4535b861 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1516 microversion: 2.1 time: 0.203496 Sep 08 13:55:54.700168 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 132/269] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:55:54 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1516 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:55:57.002259 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-14794e0e-451d-491c-8483-6a5d9c679ede tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Action: 'create', calling method: .version_select at 0x7fce022f3e50>, body: {"keypair": {"name": "tempest-keypair-1171611068", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB7maDZSr2L4iY07ZxAlWrEz8O2O3V6IALEvQbQuVcjTSgiQwyPkH0SPtRnh0iVzhHoNjceiQOdkwssvlsQnZFTbZGLhNybLEWrebBe+hsHfxY1nt4cLkuU+p5u1D31/vw=="}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:55:57.029803 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-14794e0e-451d-491c-8483-6a5d9c679ede tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Getting quotas for project 1346d4105dce4c6e928fcb11cde56854. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:55:57.037038 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-14794e0e-451d-491c-8483-6a5d9c679ede tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Getting quotas for user 792808dcbf894aa98c671cf43ac21b8b and project 1346d4105dce4c6e928fcb11cde56854. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:55:57.143762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-14794e0e-451d-491c-8483-6a5d9c679ede tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.148941 Sep 08 13:55:57.144564 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 138/270] 10.222.0.25 () {60 vars in 1159 bytes} [Thu Sep 8 13:55:56 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:55:58.349305 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0e2318fd-b453-484d-920a-4714645245d5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:55:58.350074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0e2318fd-b453-484d-920a-4714645245d5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:55:58.356103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e2318fd-b453-484d-920a-4714645245d5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:58.356738 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e2318fd-b453-484d-920a-4714645245d5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:58.357177 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e2318fd-b453-484d-920a-4714645245d5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:58.357878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e2318fd-b453-484d-920a-4714645245d5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:55:58.358349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e2318fd-b453-484d-920a-4714645245d5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:55:58.358904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e2318fd-b453-484d-920a-4714645245d5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:55:58.386933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0e2318fd-b453-484d-920a-4714645245d5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/ips" status: 200 len: 150 microversion: 2.20 time: 0.043251 Sep 08 13:55:58.388286 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 133/271] 10.222.0.25 () {60 vars in 1289 bytes} [Thu Sep 8 13:55:58 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/ips => generated 150 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:56:00.328000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-98d9daaf-d724-4cca-a23e-17596d8d4aba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-1343639252", "imageRef": "20b30425-92e5-4d78-8502-45557526ea49", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "000958f9-5cdd-4954-9b31-e6534b51d985"}], "security_groups": [{"name": "tempest-securitygroup--1314693238"}], "key_name": "tempest-keypair-1171611068", "user_data": "}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:56:00.518915 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.network.neutron [None req-98d9daaf-d724-4cca-a23e-17596d8d4aba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] validate_networks() for [('000958f9-5cdd-4954-9b31-e6534b51d985', None, None, None, None, None)] {{(pid=127602) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 13:56:01.004405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-98d9daaf-d724-4cca-a23e-17596d8d4aba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Flavor limits 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 13:56:01.004405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-98d9daaf-d724-4cca-a23e-17596d8d4aba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Image limits 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 13:56:01.004405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-98d9daaf-d724-4cca-a23e-17596d8d4aba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Flavor pref 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 13:56:01.004405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-98d9daaf-d724-4cca-a23e-17596d8d4aba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Image pref 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 13:56:01.004405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-98d9daaf-d724-4cca-a23e-17596d8d4aba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 13:56:01.021274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-98d9daaf-d724-4cca-a23e-17596d8d4aba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:01.022559 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-98d9daaf-d724-4cca-a23e-17596d8d4aba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:01.023041 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-98d9daaf-d724-4cca-a23e-17596d8d4aba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:01.023837 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-98d9daaf-d724-4cca-a23e-17596d8d4aba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:01.024200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-98d9daaf-d724-4cca-a23e-17596d8d4aba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:01.024646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-98d9daaf-d724-4cca-a23e-17596d8d4aba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:01.042930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-98d9daaf-d724-4cca-a23e-17596d8d4aba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Getting quotas for project 1346d4105dce4c6e928fcb11cde56854. Resources: {'cores', 'instances', 'ram'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:56:01.047137 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-98d9daaf-d724-4cca-a23e-17596d8d4aba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Getting quotas for user 792808dcbf894aa98c671cf43ac21b8b and project 1346d4105dce4c6e928fcb11cde56854. Resources: {'cores', 'instances', 'ram'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:56:01.061810 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-98d9daaf-d724-4cca-a23e-17596d8d4aba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Going to run 1 instances... {{(pid=127602) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 13:56:01.064240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-98d9daaf-d724-4cca-a23e-17596d8d4aba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='20b30425-92e5-4d78-8502-45557526ea49',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127602) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 13:56:01.245526 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-98d9daaf-d724-4cca-a23e-17596d8d4aba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 409 microversion: 2.1 time: 0.921791 Sep 08 13:56:01.246478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 139/272] 10.222.0.25 () {60 vars in 1147 bytes} [Thu Sep 8 13:56:00 2022] POST /compute/v2.1/servers => generated 409 bytes in 923 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 08 13:56:01.259011 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1a087f0f-2c53-4e0a-9219-db267ba08c2f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:01.259711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1a087f0f-2c53-4e0a-9219-db267ba08c2f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:01.331439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1a087f0f-2c53-4e0a-9219-db267ba08c2f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:01.343725 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1a087f0f-2c53-4e0a-9219-db267ba08c2f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1246 microversion: 2.1 time: 0.089503 Sep 08 13:56:01.345454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 134/273] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:01 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1246 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:56:02.348905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b018426e-217c-4c17-9b0c-92ccf621462c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:56:02.349446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b018426e-217c-4c17-9b0c-92ccf621462c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:02.355776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b018426e-217c-4c17-9b0c-92ccf621462c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:02.356304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b018426e-217c-4c17-9b0c-92ccf621462c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:02.356754 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b018426e-217c-4c17-9b0c-92ccf621462c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:02.357454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b018426e-217c-4c17-9b0c-92ccf621462c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:02.357903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b018426e-217c-4c17-9b0c-92ccf621462c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:02.358354 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b018426e-217c-4c17-9b0c-92ccf621462c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:02.370336 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d731c770-d51a-40ff-bd75-13eaa9115d23 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:02.377596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d731c770-d51a-40ff-bd75-13eaa9115d23 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:02.388697 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-b018426e-217c-4c17-9b0c-92ccf621462c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lazy-loading 'flavor' on Instance uuid c14f6817-e3ab-4a36-ba23-590498bf1b25 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 13:56:02.455343 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d731c770-d51a-40ff-bd75-13eaa9115d23 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:02.474633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d731c770-d51a-40ff-bd75-13eaa9115d23 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1246 microversion: 2.1 time: 0.108984 Sep 08 13:56:02.475802 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 135/274] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:02 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1246 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:56:03.094487 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b018426e-217c-4c17-9b0c-92ccf621462c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "POST /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/action" status: 202 len: 0 microversion: 2.20 time: 0.749305 Sep 08 13:56:03.095387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 140/275] 10.222.0.25 () {62 vars in 1319 bytes} [Thu Sep 8 13:56:02 2022] POST /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/action => generated 0 bytes in 752 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 08 13:56:03.105988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2774a55f-063c-42a8-93ad-77e558dca91a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:03.107765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2774a55f-063c-42a8-93ad-77e558dca91a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:03.115751 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2774a55f-063c-42a8-93ad-77e558dca91a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:03.117022 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2774a55f-063c-42a8-93ad-77e558dca91a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:03.126074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2774a55f-063c-42a8-93ad-77e558dca91a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:03.129215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2774a55f-063c-42a8-93ad-77e558dca91a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:03.130386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2774a55f-063c-42a8-93ad-77e558dca91a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:03.131212 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2774a55f-063c-42a8-93ad-77e558dca91a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:03.499174 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b2363afa-6623-4847-95f7-bad1f7d82487 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:03.500050 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b2363afa-6623-4847-95f7-bad1f7d82487 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:03.507566 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b2363afa-6623-4847-95f7-bad1f7d82487 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:03.508068 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b2363afa-6623-4847-95f7-bad1f7d82487 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:03.508563 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b2363afa-6623-4847-95f7-bad1f7d82487 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:03.509283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b2363afa-6623-4847-95f7-bad1f7d82487 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:03.510103 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b2363afa-6623-4847-95f7-bad1f7d82487 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:03.510711 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b2363afa-6623-4847-95f7-bad1f7d82487 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:03.552649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2774a55f-063c-42a8-93ad-77e558dca91a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:03.562909 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2774a55f-063c-42a8-93ad-77e558dca91a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:03.564421 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2774a55f-063c-42a8-93ad-77e558dca91a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:03.565907 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2774a55f-063c-42a8-93ad-77e558dca91a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1722 microversion: 2.20 time: 0.464716 Sep 08 13:56:03.566692 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 136/276] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:03 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1722 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:03.608727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b2363afa-6623-4847-95f7-bad1f7d82487 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:03.618952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b2363afa-6623-4847-95f7-bad1f7d82487 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1298 microversion: 2.1 time: 0.124151 Sep 08 13:56:03.626730 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 141/277] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:03 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1298 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:56:04.628885 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b5b144e4-a5de-4a01-b1ec-352f3ba5ef14 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:04.630886 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b5b144e4-a5de-4a01-b1ec-352f3ba5ef14 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:04.637951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5b144e4-a5de-4a01-b1ec-352f3ba5ef14 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:04.638636 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5b144e4-a5de-4a01-b1ec-352f3ba5ef14 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:04.639373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5b144e4-a5de-4a01-b1ec-352f3ba5ef14 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:04.640179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5b144e4-a5de-4a01-b1ec-352f3ba5ef14 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:04.641052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5b144e4-a5de-4a01-b1ec-352f3ba5ef14 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:04.642034 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5b144e4-a5de-4a01-b1ec-352f3ba5ef14 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:04.661398 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fd25df0f-436a-493b-a52d-441275eef0e7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:04.662860 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fd25df0f-436a-493b-a52d-441275eef0e7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:04.678683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fd25df0f-436a-493b-a52d-441275eef0e7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:04.680969 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fd25df0f-436a-493b-a52d-441275eef0e7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:04.681666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fd25df0f-436a-493b-a52d-441275eef0e7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:04.683698 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fd25df0f-436a-493b-a52d-441275eef0e7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:04.684187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fd25df0f-436a-493b-a52d-441275eef0e7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:04.684621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fd25df0f-436a-493b-a52d-441275eef0e7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:04.844034 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-fd25df0f-436a-493b-a52d-441275eef0e7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:04.852108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fd25df0f-436a-493b-a52d-441275eef0e7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1298 microversion: 2.1 time: 0.196023 Sep 08 13:56:04.866817 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 142/278] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:04 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1298 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:04.893019 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b5b144e4-a5de-4a01-b1ec-352f3ba5ef14 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:04.904526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b5b144e4-a5de-4a01-b1ec-352f3ba5ef14 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:04.906061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b5b144e4-a5de-4a01-b1ec-352f3ba5ef14 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:04.907283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b5b144e4-a5de-4a01-b1ec-352f3ba5ef14 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1722 microversion: 2.20 time: 0.283063 Sep 08 13:56:04.907841 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 137/279] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:04 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1722 bytes in 284 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:05.879704 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-abbcfef4-ba91-420d-a60d-dc5fb48dc3fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:05.892375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-abbcfef4-ba91-420d-a60d-dc5fb48dc3fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:05.903877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=52,nova_api:INSERT=4 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:05.909323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abbcfef4-ba91-420d-a60d-dc5fb48dc3fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:05.909579 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5a4f39fb-d75e-4859-9da8-d294f013ee1a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:05.909579 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5a4f39fb-d75e-4859-9da8-d294f013ee1a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:05.910076 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abbcfef4-ba91-420d-a60d-dc5fb48dc3fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:05.910076 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abbcfef4-ba91-420d-a60d-dc5fb48dc3fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:05.910076 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abbcfef4-ba91-420d-a60d-dc5fb48dc3fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:05.910346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abbcfef4-ba91-420d-a60d-dc5fb48dc3fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:05.910893 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abbcfef4-ba91-420d-a60d-dc5fb48dc3fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:05.922423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a4f39fb-d75e-4859-9da8-d294f013ee1a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:05.924676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a4f39fb-d75e-4859-9da8-d294f013ee1a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:05.925326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a4f39fb-d75e-4859-9da8-d294f013ee1a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:05.930823 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a4f39fb-d75e-4859-9da8-d294f013ee1a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:05.931572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a4f39fb-d75e-4859-9da8-d294f013ee1a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:05.932278 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a4f39fb-d75e-4859-9da8-d294f013ee1a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:05.989606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5a4f39fb-d75e-4859-9da8-d294f013ee1a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/ips" status: 200 len: 149 microversion: 2.1 time: 0.085630 Sep 08 13:56:05.991006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 138/280] 10.222.0.25 () {58 vars in 1248 bytes} [Thu Sep 8 13:56:05 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/ips => generated 149 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:56:06.005108 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-71060d15-48f5-46e5-b302-6da506c872e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:06.005741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-71060d15-48f5-46e5-b302-6da506c872e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:06.021927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-71060d15-48f5-46e5-b302-6da506c872e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:06.023705 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-71060d15-48f5-46e5-b302-6da506c872e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:06.024644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-71060d15-48f5-46e5-b302-6da506c872e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:06.028028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-71060d15-48f5-46e5-b302-6da506c872e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:06.029151 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-71060d15-48f5-46e5-b302-6da506c872e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:06.030423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-71060d15-48f5-46e5-b302-6da506c872e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:06.447299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-71060d15-48f5-46e5-b302-6da506c872e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:06.447560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-abbcfef4-ba91-420d-a60d-dc5fb48dc3fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:06.459517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-71060d15-48f5-46e5-b302-6da506c872e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:06.463400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-71060d15-48f5-46e5-b302-6da506c872e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:06.465437 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-abbcfef4-ba91-420d-a60d-dc5fb48dc3fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1304 microversion: 2.1 time: 0.593865 Sep 08 13:56:06.466878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-71060d15-48f5-46e5-b302-6da506c872e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1722 microversion: 2.20 time: 0.473068 Sep 08 13:56:06.468196 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 143/281] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:05 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1304 bytes in 597 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:06.468917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 139/282] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:05 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1722 bytes in 477 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:07.492350 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-614b6e1c-cc04-46aa-a728-f8f9810c926e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:07.493061 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-614b6e1c-cc04-46aa-a728-f8f9810c926e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:07.498880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0e2c4d19-cfaa-4efb-9d58-fe23426237f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:07.500064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0e2c4d19-cfaa-4efb-9d58-fe23426237f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:07.519820 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e2c4d19-cfaa-4efb-9d58-fe23426237f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:07.525400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e2c4d19-cfaa-4efb-9d58-fe23426237f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:07.526334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e2c4d19-cfaa-4efb-9d58-fe23426237f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:07.527712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e2c4d19-cfaa-4efb-9d58-fe23426237f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:07.528354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e2c4d19-cfaa-4efb-9d58-fe23426237f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:07.528580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-614b6e1c-cc04-46aa-a728-f8f9810c926e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:07.529365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e2c4d19-cfaa-4efb-9d58-fe23426237f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:07.529789 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-614b6e1c-cc04-46aa-a728-f8f9810c926e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:07.531024 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-614b6e1c-cc04-46aa-a728-f8f9810c926e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:07.537836 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-614b6e1c-cc04-46aa-a728-f8f9810c926e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:07.538398 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-614b6e1c-cc04-46aa-a728-f8f9810c926e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:07.541869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-614b6e1c-cc04-46aa-a728-f8f9810c926e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:07.555768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:08.125367 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0e2c4d19-cfaa-4efb-9d58-fe23426237f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:08.158072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0e2c4d19-cfaa-4efb-9d58-fe23426237f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:08.173558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0e2c4d19-cfaa-4efb-9d58-fe23426237f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:08.175359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0e2c4d19-cfaa-4efb-9d58-fe23426237f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1722 microversion: 2.20 time: 0.683400 Sep 08 13:56:08.202225 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 140/283] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:07 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1722 bytes in 708 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:56:08.352007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-614b6e1c-cc04-46aa-a728-f8f9810c926e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:08.362422 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-614b6e1c-cc04-46aa-a728-f8f9810c926e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1372 microversion: 2.1 time: 0.876767 Sep 08 13:56:08.375038 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 144/284] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:07 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1372 bytes in 890 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:09.213266 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-82ca1555-8178-4acc-bbb8-440f8d5a36e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:09.213942 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-82ca1555-8178-4acc-bbb8-440f8d5a36e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:09.220945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-82ca1555-8178-4acc-bbb8-440f8d5a36e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:09.221445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-82ca1555-8178-4acc-bbb8-440f8d5a36e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:09.221881 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-82ca1555-8178-4acc-bbb8-440f8d5a36e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:09.222829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-82ca1555-8178-4acc-bbb8-440f8d5a36e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:09.223290 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-82ca1555-8178-4acc-bbb8-440f8d5a36e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:09.223919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-82ca1555-8178-4acc-bbb8-440f8d5a36e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:09.387138 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8a808d35-acfb-46ba-b7e1-5beac47e7069 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:09.388402 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8a808d35-acfb-46ba-b7e1-5beac47e7069 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:09.399302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8a808d35-acfb-46ba-b7e1-5beac47e7069 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:09.401488 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8a808d35-acfb-46ba-b7e1-5beac47e7069 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:09.404044 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8a808d35-acfb-46ba-b7e1-5beac47e7069 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:09.405010 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8a808d35-acfb-46ba-b7e1-5beac47e7069 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:09.407219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8a808d35-acfb-46ba-b7e1-5beac47e7069 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:09.409586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8a808d35-acfb-46ba-b7e1-5beac47e7069 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:09.797472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-82ca1555-8178-4acc-bbb8-440f8d5a36e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:09.809715 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-82ca1555-8178-4acc-bbb8-440f8d5a36e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:09.816499 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-82ca1555-8178-4acc-bbb8-440f8d5a36e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:09.818731 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-82ca1555-8178-4acc-bbb8-440f8d5a36e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1722 microversion: 2.20 time: 0.614219 Sep 08 13:56:09.820838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 141/285] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:09 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1722 bytes in 617 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:09.834337 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8a808d35-acfb-46ba-b7e1-5beac47e7069 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:09.834741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6cc6dba6-880d-470e-8fa7-3bbf9398b637 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "06ab5239-fc50-4a85-aa7f-2e9dbf1b3e9d"}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:56:09.839255 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6cc6dba6-880d-470e-8fa7-3bbf9398b637 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:09.857127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8a808d35-acfb-46ba-b7e1-5beac47e7069 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1372 microversion: 2.1 time: 0.471673 Sep 08 13:56:09.859018 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6cc6dba6-880d-470e-8fa7-3bbf9398b637 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:09.867169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6cc6dba6-880d-470e-8fa7-3bbf9398b637 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:09.867169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6cc6dba6-880d-470e-8fa7-3bbf9398b637 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:09.867169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6cc6dba6-880d-470e-8fa7-3bbf9398b637 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:09.867533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 145/286] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:09 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1372 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:56:09.867619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6cc6dba6-880d-470e-8fa7-3bbf9398b637 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:09.867619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6cc6dba6-880d-470e-8fa7-3bbf9398b637 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:10.053524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-a402dcd1-86c2-43fe-8cb8-e88921074ec4 req-96d10b9f-1a82-4483-bdb0-50ffc51bd684 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "19db4aa2-f574-43fa-bbf8-c3042b243152", "tag": "2ebf3a50-af15-4ecc-820b-87ad0a9ff652"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:56:10.071928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-a402dcd1-86c2-43fe-8cb8-e88921074ec4 req-96d10b9f-1a82-4483-bdb0-50ffc51bd684 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:10.075483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-a402dcd1-86c2-43fe-8cb8-e88921074ec4 req-96d10b9f-1a82-4483-bdb0-50ffc51bd684 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:10.076563 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-a402dcd1-86c2-43fe-8cb8-e88921074ec4 req-96d10b9f-1a82-4483-bdb0-50ffc51bd684 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:10.097209 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-a402dcd1-86c2-43fe-8cb8-e88921074ec4 req-96d10b9f-1a82-4483-bdb0-50ffc51bd684 service nova] Creating event network-changed:2ebf3a50-af15-4ecc-820b-87ad0a9ff652 for instance 19db4aa2-f574-43fa-bbf8-c3042b243152 on hv-cinder-85646 Sep 08 13:56:10.102730 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-a402dcd1-86c2-43fe-8cb8-e88921074ec4 req-96d10b9f-1a82-4483-bdb0-50ffc51bd684 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054671 Sep 08 13:56:10.103642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 146/287] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:56:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:56:10.860140 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-79cf475e-5fec-492f-9640-6ccd1ef4a54e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:10.863565 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-79cf475e-5fec-492f-9640-6ccd1ef4a54e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:10.883742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79cf475e-5fec-492f-9640-6ccd1ef4a54e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:10.901268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79cf475e-5fec-492f-9640-6ccd1ef4a54e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:10.902642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79cf475e-5fec-492f-9640-6ccd1ef4a54e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:10.912786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79cf475e-5fec-492f-9640-6ccd1ef4a54e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:10.913403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79cf475e-5fec-492f-9640-6ccd1ef4a54e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:10.916575 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79cf475e-5fec-492f-9640-6ccd1ef4a54e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:11.034093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=2 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:11.055145 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=10 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:11.315783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6cc6dba6-880d-470e-8fa7-3bbf9398b637 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "POST /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.492440 Sep 08 13:56:11.323734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 142/288] 10.222.0.25 () {60 vars in 1323 bytes} [Thu Sep 8 13:56:09 2022] POST /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments => generated 194 bytes in 1495 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:56:11.341899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3218c6ec-df80-45c9-9bdb-1d7b5b2471e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:11.341899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3218c6ec-df80-45c9-9bdb-1d7b5b2471e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:11.362765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3218c6ec-df80-45c9-9bdb-1d7b5b2471e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:11.363267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3218c6ec-df80-45c9-9bdb-1d7b5b2471e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:11.366946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3218c6ec-df80-45c9-9bdb-1d7b5b2471e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:11.371993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3218c6ec-df80-45c9-9bdb-1d7b5b2471e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:11.372716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3218c6ec-df80-45c9-9bdb-1d7b5b2471e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:11.373163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3218c6ec-df80-45c9-9bdb-1d7b5b2471e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:11.409486 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:11.441072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-79cf475e-5fec-492f-9640-6ccd1ef4a54e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:11.450111 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-79cf475e-5fec-492f-9640-6ccd1ef4a54e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:11.451484 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-79cf475e-5fec-492f-9640-6ccd1ef4a54e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:11.452691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-79cf475e-5fec-492f-9640-6ccd1ef4a54e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1722 microversion: 2.20 time: 0.600813 Sep 08 13:56:11.453264 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 147/289] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:10 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1722 bytes in 602 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:11.838856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3218c6ec-df80-45c9-9bdb-1d7b5b2471e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:11.849755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3218c6ec-df80-45c9-9bdb-1d7b5b2471e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1372 microversion: 2.1 time: 0.524272 Sep 08 13:56:11.866165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 143/290] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:11 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1372 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:12.483618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b5f389ff-9a4c-4941-8496-5e1039e4f30e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:12.484286 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b5f389ff-9a4c-4941-8496-5e1039e4f30e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:12.493613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b5f389ff-9a4c-4941-8496-5e1039e4f30e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:12.494724 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b5f389ff-9a4c-4941-8496-5e1039e4f30e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:12.495255 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b5f389ff-9a4c-4941-8496-5e1039e4f30e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:12.496114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b5f389ff-9a4c-4941-8496-5e1039e4f30e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:12.496648 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b5f389ff-9a4c-4941-8496-5e1039e4f30e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:12.497226 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b5f389ff-9a4c-4941-8496-5e1039e4f30e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:12.881992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-77305721-19fd-4c66-ae85-0dd2b0aa714d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:12.882648 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-77305721-19fd-4c66-ae85-0dd2b0aa714d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:12.887498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=4 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:12.903654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77305721-19fd-4c66-ae85-0dd2b0aa714d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:12.903654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77305721-19fd-4c66-ae85-0dd2b0aa714d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:12.903654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77305721-19fd-4c66-ae85-0dd2b0aa714d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:12.903654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77305721-19fd-4c66-ae85-0dd2b0aa714d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:12.904197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77305721-19fd-4c66-ae85-0dd2b0aa714d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:12.904637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77305721-19fd-4c66-ae85-0dd2b0aa714d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:13.137758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-77305721-19fd-4c66-ae85-0dd2b0aa714d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:13.177854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-77305721-19fd-4c66-ae85-0dd2b0aa714d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1372 microversion: 2.1 time: 0.276805 Sep 08 13:56:13.177854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 144/291] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:12 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1372 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:13.238651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b5f389ff-9a4c-4941-8496-5e1039e4f30e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:13.248535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b5f389ff-9a4c-4941-8496-5e1039e4f30e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:13.250627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b5f389ff-9a4c-4941-8496-5e1039e4f30e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:13.252116 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b5f389ff-9a4c-4941-8496-5e1039e4f30e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1722 microversion: 2.20 time: 0.772673 Sep 08 13:56:13.252691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 148/292] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:12 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1722 bytes in 774 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:14.190043 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1ea9b03d-1b03-4161-bdef-0c90e5db356a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:14.191518 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1ea9b03d-1b03-4161-bdef-0c90e5db356a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:14.203759 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1ea9b03d-1b03-4161-bdef-0c90e5db356a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:14.207136 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1ea9b03d-1b03-4161-bdef-0c90e5db356a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:14.209409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1ea9b03d-1b03-4161-bdef-0c90e5db356a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:14.210712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1ea9b03d-1b03-4161-bdef-0c90e5db356a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:14.212140 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1ea9b03d-1b03-4161-bdef-0c90e5db356a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:14.214199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1ea9b03d-1b03-4161-bdef-0c90e5db356a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:14.285939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ee8064fc-3c48-44a0-9e34-a655f26030cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:14.285939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ee8064fc-3c48-44a0-9e34-a655f26030cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:14.298374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ee8064fc-3c48-44a0-9e34-a655f26030cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:14.299030 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ee8064fc-3c48-44a0-9e34-a655f26030cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:14.299486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ee8064fc-3c48-44a0-9e34-a655f26030cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:14.300470 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ee8064fc-3c48-44a0-9e34-a655f26030cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:14.301005 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ee8064fc-3c48-44a0-9e34-a655f26030cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:14.307852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ee8064fc-3c48-44a0-9e34-a655f26030cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:14.640252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ee8064fc-3c48-44a0-9e34-a655f26030cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:14.659618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ee8064fc-3c48-44a0-9e34-a655f26030cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:14.663269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ee8064fc-3c48-44a0-9e34-a655f26030cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:14.665104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ee8064fc-3c48-44a0-9e34-a655f26030cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1722 microversion: 2.20 time: 0.388356 Sep 08 13:56:14.666301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 149/293] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:14 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1722 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:56:14.734624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1ea9b03d-1b03-4161-bdef-0c90e5db356a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:14.750076 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1ea9b03d-1b03-4161-bdef-0c90e5db356a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1372 microversion: 2.1 time: 0.562706 Sep 08 13:56:14.757366 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 145/294] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:14 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1372 bytes in 572 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:15.710928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a263cdf7-4caa-421c-b662-c912ad98788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:15.711538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a263cdf7-4caa-421c-b662-c912ad98788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:15.731404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a263cdf7-4caa-421c-b662-c912ad98788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:15.736468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a263cdf7-4caa-421c-b662-c912ad98788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:15.736468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a263cdf7-4caa-421c-b662-c912ad98788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:15.736468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a263cdf7-4caa-421c-b662-c912ad98788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:15.736468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a263cdf7-4caa-421c-b662-c912ad98788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:15.736970 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a263cdf7-4caa-421c-b662-c912ad98788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:15.789926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-60b2bafc-4b9a-4757-b58f-b69603e15c93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:15.794097 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-60b2bafc-4b9a-4757-b58f-b69603e15c93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:15.802486 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-60b2bafc-4b9a-4757-b58f-b69603e15c93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:15.803677 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-60b2bafc-4b9a-4757-b58f-b69603e15c93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:15.805436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-60b2bafc-4b9a-4757-b58f-b69603e15c93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:15.806471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-60b2bafc-4b9a-4757-b58f-b69603e15c93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:15.807202 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-60b2bafc-4b9a-4757-b58f-b69603e15c93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:15.808423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-60b2bafc-4b9a-4757-b58f-b69603e15c93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:15.984745 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a263cdf7-4caa-421c-b662-c912ad98788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:15.995820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a263cdf7-4caa-421c-b662-c912ad98788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:16.001170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a263cdf7-4caa-421c-b662-c912ad98788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:16.002547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a263cdf7-4caa-421c-b662-c912ad98788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1722 microversion: 2.20 time: 0.304528 Sep 08 13:56:16.008841 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 150/295] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:15 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1722 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:16.170641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-60b2bafc-4b9a-4757-b58f-b69603e15c93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:16.179347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-60b2bafc-4b9a-4757-b58f-b69603e15c93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1372 microversion: 2.1 time: 0.393851 Sep 08 13:56:16.179970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 146/296] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:15 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1372 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:17.039234 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-79d95598-40a3-4e36-b623-8c5194477314 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:17.040468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-79d95598-40a3-4e36-b623-8c5194477314 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:17.052765 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79d95598-40a3-4e36-b623-8c5194477314 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:17.053912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79d95598-40a3-4e36-b623-8c5194477314 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:17.055081 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79d95598-40a3-4e36-b623-8c5194477314 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:17.055810 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79d95598-40a3-4e36-b623-8c5194477314 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:17.058898 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79d95598-40a3-4e36-b623-8c5194477314 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:17.059353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79d95598-40a3-4e36-b623-8c5194477314 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:17.207397 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ce8f0a4d-4a69-4146-bdfe-3a700ec5e102 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:17.210609 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ce8f0a4d-4a69-4146-bdfe-3a700ec5e102 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:17.233009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ce8f0a4d-4a69-4146-bdfe-3a700ec5e102 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:17.236244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ce8f0a4d-4a69-4146-bdfe-3a700ec5e102 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:17.236845 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ce8f0a4d-4a69-4146-bdfe-3a700ec5e102 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:17.239941 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ce8f0a4d-4a69-4146-bdfe-3a700ec5e102 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:17.240631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ce8f0a4d-4a69-4146-bdfe-3a700ec5e102 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:17.241073 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ce8f0a4d-4a69-4146-bdfe-3a700ec5e102 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:17.600791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-79d95598-40a3-4e36-b623-8c5194477314 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:17.602193 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ce8f0a4d-4a69-4146-bdfe-3a700ec5e102 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:17.627258 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ce8f0a4d-4a69-4146-bdfe-3a700ec5e102 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1372 microversion: 2.1 time: 0.423321 Sep 08 13:56:17.628380 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-79d95598-40a3-4e36-b623-8c5194477314 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:17.633279 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-79d95598-40a3-4e36-b623-8c5194477314 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:17.638045 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-79d95598-40a3-4e36-b623-8c5194477314 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1722 microversion: 2.20 time: 0.603181 Sep 08 13:56:17.658171 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 151/297] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:17 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1722 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:17.661716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 147/298] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:17 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1372 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:56:18.670692 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f1d3bf85-7b3d-45cf-8f22-ed609731f094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:18.671281 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f1d3bf85-7b3d-45cf-8f22-ed609731f094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:18.681303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a13dd346-1a4b-4cf7-8c8b-848bbeb6d0bd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:18.682095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a13dd346-1a4b-4cf7-8c8b-848bbeb6d0bd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:18.683403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f1d3bf85-7b3d-45cf-8f22-ed609731f094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:18.684121 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f1d3bf85-7b3d-45cf-8f22-ed609731f094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:18.684561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f1d3bf85-7b3d-45cf-8f22-ed609731f094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:18.685305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f1d3bf85-7b3d-45cf-8f22-ed609731f094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:18.685800 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f1d3bf85-7b3d-45cf-8f22-ed609731f094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:18.686213 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f1d3bf85-7b3d-45cf-8f22-ed609731f094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:18.699033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a13dd346-1a4b-4cf7-8c8b-848bbeb6d0bd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:18.703647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a13dd346-1a4b-4cf7-8c8b-848bbeb6d0bd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:18.704264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a13dd346-1a4b-4cf7-8c8b-848bbeb6d0bd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:18.706670 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a13dd346-1a4b-4cf7-8c8b-848bbeb6d0bd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:18.707253 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a13dd346-1a4b-4cf7-8c8b-848bbeb6d0bd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:18.707784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a13dd346-1a4b-4cf7-8c8b-848bbeb6d0bd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:19.119688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a13dd346-1a4b-4cf7-8c8b-848bbeb6d0bd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:19.212750 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a13dd346-1a4b-4cf7-8c8b-848bbeb6d0bd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:19.219275 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f1d3bf85-7b3d-45cf-8f22-ed609731f094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:19.225331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a13dd346-1a4b-4cf7-8c8b-848bbeb6d0bd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:19.233641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a13dd346-1a4b-4cf7-8c8b-848bbeb6d0bd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1722 microversion: 2.20 time: 0.556729 Sep 08 13:56:19.241265 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 148/299] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:18 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1722 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:19.255848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f1d3bf85-7b3d-45cf-8f22-ed609731f094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1372 microversion: 2.1 time: 0.589944 Sep 08 13:56:19.261577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 152/300] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:18 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1372 bytes in 592 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:20.287492 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0aa7bd1c-3133-4538-8048-efba1b36edb3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:20.292422 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0aa7bd1c-3133-4538-8048-efba1b36edb3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:20.298339 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0d751429-794c-4878-aee5-03b1158aa3b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:20.299744 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0d751429-794c-4878-aee5-03b1158aa3b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:20.299873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0aa7bd1c-3133-4538-8048-efba1b36edb3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:20.302768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0aa7bd1c-3133-4538-8048-efba1b36edb3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:20.303451 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0aa7bd1c-3133-4538-8048-efba1b36edb3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:20.304268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0aa7bd1c-3133-4538-8048-efba1b36edb3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:20.307709 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0aa7bd1c-3133-4538-8048-efba1b36edb3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:20.308526 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0aa7bd1c-3133-4538-8048-efba1b36edb3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:20.312458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=42 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:20.315460 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d751429-794c-4878-aee5-03b1158aa3b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:20.316534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d751429-794c-4878-aee5-03b1158aa3b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:20.317180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d751429-794c-4878-aee5-03b1158aa3b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:20.320004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d751429-794c-4878-aee5-03b1158aa3b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:20.320742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d751429-794c-4878-aee5-03b1158aa3b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:20.321406 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d751429-794c-4878-aee5-03b1158aa3b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:20.702829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0aa7bd1c-3133-4538-8048-efba1b36edb3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:20.713310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0aa7bd1c-3133-4538-8048-efba1b36edb3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:20.717690 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0aa7bd1c-3133-4538-8048-efba1b36edb3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:20.736601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0aa7bd1c-3133-4538-8048-efba1b36edb3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1722 microversion: 2.20 time: 0.453872 Sep 08 13:56:20.737329 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 153/301] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:20 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1722 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:21.092484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0d751429-794c-4878-aee5-03b1158aa3b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:21.108829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0d751429-794c-4878-aee5-03b1158aa3b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1372 microversion: 2.1 time: 0.825256 Sep 08 13:56:21.124828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 149/302] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:20 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1372 bytes in 843 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:21.765778 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-274798d5-841a-4d45-9d1e-d35c11e3d0c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:21.767181 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-274798d5-841a-4d45-9d1e-d35c11e3d0c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:21.791230 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-274798d5-841a-4d45-9d1e-d35c11e3d0c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:21.792104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-274798d5-841a-4d45-9d1e-d35c11e3d0c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:21.798967 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-274798d5-841a-4d45-9d1e-d35c11e3d0c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:21.800872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-274798d5-841a-4d45-9d1e-d35c11e3d0c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:21.803220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-274798d5-841a-4d45-9d1e-d35c11e3d0c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:21.803879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-274798d5-841a-4d45-9d1e-d35c11e3d0c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:22.150686 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-648bc2d0-7df5-4846-9515-b7d9e8932bcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:22.150686 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-648bc2d0-7df5-4846-9515-b7d9e8932bcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:22.157514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-648bc2d0-7df5-4846-9515-b7d9e8932bcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:22.158572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-648bc2d0-7df5-4846-9515-b7d9e8932bcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:22.159385 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-648bc2d0-7df5-4846-9515-b7d9e8932bcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:22.162557 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-648bc2d0-7df5-4846-9515-b7d9e8932bcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:22.164059 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-648bc2d0-7df5-4846-9515-b7d9e8932bcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:22.168866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-648bc2d0-7df5-4846-9515-b7d9e8932bcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:22.240992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-274798d5-841a-4d45-9d1e-d35c11e3d0c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:22.265056 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-274798d5-841a-4d45-9d1e-d35c11e3d0c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:22.267110 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-274798d5-841a-4d45-9d1e-d35c11e3d0c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:22.268612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-274798d5-841a-4d45-9d1e-d35c11e3d0c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1722 microversion: 2.20 time: 0.507674 Sep 08 13:56:22.269613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 154/303] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:21 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1722 bytes in 509 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:22.323849 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-5fb94585-5547-4bc8-b2bb-2f420ea9bb62 req-544f606c-8158-459a-935b-53e6ea901b84 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "19db4aa2-f574-43fa-bbf8-c3042b243152", "name": "network-vif-plugged", "status": "completed", "tag": "2ebf3a50-af15-4ecc-820b-87ad0a9ff652"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:56:22.347584 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-5fb94585-5547-4bc8-b2bb-2f420ea9bb62 req-544f606c-8158-459a-935b-53e6ea901b84 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:22.347584 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-5fb94585-5547-4bc8-b2bb-2f420ea9bb62 req-544f606c-8158-459a-935b-53e6ea901b84 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:22.347584 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-5fb94585-5547-4bc8-b2bb-2f420ea9bb62 req-544f606c-8158-459a-935b-53e6ea901b84 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:22.367188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-5fb94585-5547-4bc8-b2bb-2f420ea9bb62 req-544f606c-8158-459a-935b-53e6ea901b84 service nova] Creating event network-vif-plugged:2ebf3a50-af15-4ecc-820b-87ad0a9ff652 for instance 19db4aa2-f574-43fa-bbf8-c3042b243152 on hv-cinder-85646 Sep 08 13:56:22.379022 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-5fb94585-5547-4bc8-b2bb-2f420ea9bb62 req-544f606c-8158-459a-935b-53e6ea901b84 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059140 Sep 08 13:56:22.380125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 155/304] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:56:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:56:23.192393 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-648bc2d0-7df5-4846-9515-b7d9e8932bcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:23.210531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-648bc2d0-7df5-4846-9515-b7d9e8932bcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1372 microversion: 2.1 time: 1.066986 Sep 08 13:56:23.211211 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 150/305] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:22 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1372 bytes in 1077 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:23.294734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a34c4266-d1c6-4dc5-9233-3ed42674f9d9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Action: 'create', calling method: .version_select at 0x7fce023b3040>, body: {"keypair": {"name": "tempest-keypair-52760673", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMTiGIlBmr0ndExDtr0tNigfDbZzN2g1EFVspiXS/70bcB7KUDY7qoVwfXN1lZBR62PrNTv82PmPZew1zpi6OFvSp3drlFpjoiIlcI5dyZKPu5yRs9PdzZNc/pksbffJYA=="}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:56:23.306958 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-38072b78-6ba1-4b81-bfc6-a93c36d8228e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:23.306958 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-38072b78-6ba1-4b81-bfc6-a93c36d8228e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:23.322221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38072b78-6ba1-4b81-bfc6-a93c36d8228e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:23.324988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38072b78-6ba1-4b81-bfc6-a93c36d8228e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:23.331755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38072b78-6ba1-4b81-bfc6-a93c36d8228e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:23.333814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38072b78-6ba1-4b81-bfc6-a93c36d8228e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:23.334341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38072b78-6ba1-4b81-bfc6-a93c36d8228e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:23.338573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38072b78-6ba1-4b81-bfc6-a93c36d8228e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:23.385443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-a34c4266-d1c6-4dc5-9233-3ed42674f9d9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Getting quotas for project a4e0fba6cab34c429c73fcfac7ebfd94. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:56:23.391372 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-a34c4266-d1c6-4dc5-9233-3ed42674f9d9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Getting quotas for user eca89f435299415c8448931e73f5b0fc and project a4e0fba6cab34c429c73fcfac7ebfd94. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:56:23.540092 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a34c4266-d1c6-4dc5-9233-3ed42674f9d9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.271356 Sep 08 13:56:23.541473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 156/306] 10.222.0.25 () {60 vars in 1159 bytes} [Thu Sep 8 13:56:23 2022] POST /compute/v2.1/os-keypairs => generated 384 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:56:23.670734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-38072b78-6ba1-4b81-bfc6-a93c36d8228e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:23.681070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-38072b78-6ba1-4b81-bfc6-a93c36d8228e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:23.683019 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-38072b78-6ba1-4b81-bfc6-a93c36d8228e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:23.684400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-38072b78-6ba1-4b81-bfc6-a93c36d8228e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1722 microversion: 2.20 time: 0.392259 Sep 08 13:56:23.685191 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 151/307] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:23 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1722 bytes in 394 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:56:24.233417 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4e15b2b1-5b9e-43f5-8e6f-3465b59358e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:24.238983 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4e15b2b1-5b9e-43f5-8e6f-3465b59358e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:24.243268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4e15b2b1-5b9e-43f5-8e6f-3465b59358e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:24.248999 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4e15b2b1-5b9e-43f5-8e6f-3465b59358e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:24.249733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4e15b2b1-5b9e-43f5-8e6f-3465b59358e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:24.251174 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4e15b2b1-5b9e-43f5-8e6f-3465b59358e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:24.253674 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4e15b2b1-5b9e-43f5-8e6f-3465b59358e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:24.255688 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4e15b2b1-5b9e-43f5-8e6f-3465b59358e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:24.548131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4e15b2b1-5b9e-43f5-8e6f-3465b59358e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:24.578693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4e15b2b1-5b9e-43f5-8e6f-3465b59358e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1372 microversion: 2.1 time: 0.331089 Sep 08 13:56:24.604064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 157/308] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:24 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1372 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:24.717261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6197af5b-41c9-456a-a5e0-cb9f4143768c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:24.718931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6197af5b-41c9-456a-a5e0-cb9f4143768c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:24.735737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6197af5b-41c9-456a-a5e0-cb9f4143768c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:24.736940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6197af5b-41c9-456a-a5e0-cb9f4143768c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:24.737945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6197af5b-41c9-456a-a5e0-cb9f4143768c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:24.739432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6197af5b-41c9-456a-a5e0-cb9f4143768c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:24.740091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6197af5b-41c9-456a-a5e0-cb9f4143768c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:24.740676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6197af5b-41c9-456a-a5e0-cb9f4143768c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:25.291067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6197af5b-41c9-456a-a5e0-cb9f4143768c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:25.330393 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6197af5b-41c9-456a-a5e0-cb9f4143768c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:25.331890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6197af5b-41c9-456a-a5e0-cb9f4143768c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:25.333367 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6197af5b-41c9-456a-a5e0-cb9f4143768c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1722 microversion: 2.20 time: 0.620536 Sep 08 13:56:25.356917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 152/309] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:24 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1722 bytes in 645 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:25.628626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-93f3c5c6-f08f-4fc0-89d8-8716662ff1e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:25.630636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-93f3c5c6-f08f-4fc0-89d8-8716662ff1e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:25.647871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-93f3c5c6-f08f-4fc0-89d8-8716662ff1e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:25.648385 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-93f3c5c6-f08f-4fc0-89d8-8716662ff1e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:25.649266 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-93f3c5c6-f08f-4fc0-89d8-8716662ff1e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:25.650317 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-93f3c5c6-f08f-4fc0-89d8-8716662ff1e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:25.651185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-93f3c5c6-f08f-4fc0-89d8-8716662ff1e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:25.669783 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-93f3c5c6-f08f-4fc0-89d8-8716662ff1e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:25.978660 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-93f3c5c6-f08f-4fc0-89d8-8716662ff1e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:25.990697 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-93f3c5c6-f08f-4fc0-89d8-8716662ff1e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1372 microversion: 2.1 time: 0.379071 Sep 08 13:56:26.006951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 158/310] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:25 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1372 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:26.370112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-40c9027b-349e-49b8-be7a-e868cbe4665c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:26.370112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-40c9027b-349e-49b8-be7a-e868cbe4665c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:26.378214 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-40c9027b-349e-49b8-be7a-e868cbe4665c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:26.378662 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-40c9027b-349e-49b8-be7a-e868cbe4665c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:26.379087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-40c9027b-349e-49b8-be7a-e868cbe4665c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:26.379816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-40c9027b-349e-49b8-be7a-e868cbe4665c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:26.381607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-40c9027b-349e-49b8-be7a-e868cbe4665c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:26.386037 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-40c9027b-349e-49b8-be7a-e868cbe4665c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:26.817210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-40c9027b-349e-49b8-be7a-e868cbe4665c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:26.835158 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-40c9027b-349e-49b8-be7a-e868cbe4665c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:26.839622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-40c9027b-349e-49b8-be7a-e868cbe4665c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:26.839622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-40c9027b-349e-49b8-be7a-e868cbe4665c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1743 microversion: 2.20 time: 0.483132 Sep 08 13:56:26.841311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 153/311] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:26 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1743 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:27.014662 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c8d3071f-04b9-4b7f-a3ba-33c04c9a77ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:27.015283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c8d3071f-04b9-4b7f-a3ba-33c04c9a77ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:27.030995 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c8d3071f-04b9-4b7f-a3ba-33c04c9a77ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:27.032761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c8d3071f-04b9-4b7f-a3ba-33c04c9a77ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:27.038864 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c8d3071f-04b9-4b7f-a3ba-33c04c9a77ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:27.039993 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c8d3071f-04b9-4b7f-a3ba-33c04c9a77ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:27.040837 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c8d3071f-04b9-4b7f-a3ba-33c04c9a77ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:27.041499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c8d3071f-04b9-4b7f-a3ba-33c04c9a77ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:27.334627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c8d3071f-04b9-4b7f-a3ba-33c04c9a77ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:27.354009 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c8d3071f-04b9-4b7f-a3ba-33c04c9a77ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1565 microversion: 2.1 time: 0.341363 Sep 08 13:56:27.377224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 159/312] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:27 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1565 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:27.398004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1d61532d-e275-479a-b0d4-288346cd038e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:27.402413 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1d61532d-e275-479a-b0d4-288346cd038e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:27.425238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d61532d-e275-479a-b0d4-288346cd038e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:27.425238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d61532d-e275-479a-b0d4-288346cd038e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:27.425238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d61532d-e275-479a-b0d4-288346cd038e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:27.425238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d61532d-e275-479a-b0d4-288346cd038e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:27.425979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d61532d-e275-479a-b0d4-288346cd038e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:27.425979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d61532d-e275-479a-b0d4-288346cd038e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:27.750971 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1d61532d-e275-479a-b0d4-288346cd038e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.351068 Sep 08 13:56:27.753632 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 154/313] 10.222.0.25 () {58 vars in 1275 bytes} [Thu Sep 8 13:56:27 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-interface => generated 284 bytes in 361 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:56:27.866348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fceba1d5-fcad-4516-9f56-33320afe6ecf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:27.867063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fceba1d5-fcad-4516-9f56-33320afe6ecf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:27.882523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fceba1d5-fcad-4516-9f56-33320afe6ecf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:27.882935 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fceba1d5-fcad-4516-9f56-33320afe6ecf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:27.883692 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fceba1d5-fcad-4516-9f56-33320afe6ecf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:27.884589 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fceba1d5-fcad-4516-9f56-33320afe6ecf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:27.886338 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fceba1d5-fcad-4516-9f56-33320afe6ecf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:27.889511 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fceba1d5-fcad-4516-9f56-33320afe6ecf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:28.271093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-fceba1d5-fcad-4516-9f56-33320afe6ecf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:28.282494 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-fceba1d5-fcad-4516-9f56-33320afe6ecf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:28.284074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-fceba1d5-fcad-4516-9f56-33320afe6ecf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:28.285291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fceba1d5-fcad-4516-9f56-33320afe6ecf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1743 microversion: 2.20 time: 0.423134 Sep 08 13:56:28.298923 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 160/314] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:27 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1743 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:28.325053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-877ca352-4c6d-4552-adc0-115b4afec82e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-898198039", "imageRef": "20b30425-92e5-4d78-8502-45557526ea49", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "a265cc60-700e-4951-9295-70d02eb6ee91"}], "security_groups": [{"name": "tempest-securitygroup--1776092173"}], "key_name": "tempest-keypair-52760673", "user_data": "}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:56:28.813940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.network.neutron [None req-877ca352-4c6d-4552-adc0-115b4afec82e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] validate_networks() for [('a265cc60-700e-4951-9295-70d02eb6ee91', None, None, None, None, None)] {{(pid=127601) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 13:56:29.320672 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-70bd8e77-6baf-495f-a2fc-2495633716fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:29.331009 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-70bd8e77-6baf-495f-a2fc-2495633716fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:29.335742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-70bd8e77-6baf-495f-a2fc-2495633716fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:29.336523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-70bd8e77-6baf-495f-a2fc-2495633716fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:29.337828 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-70bd8e77-6baf-495f-a2fc-2495633716fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:29.338571 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-70bd8e77-6baf-495f-a2fc-2495633716fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:29.341342 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-70bd8e77-6baf-495f-a2fc-2495633716fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:29.342020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-70bd8e77-6baf-495f-a2fc-2495633716fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:29.969110 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-70bd8e77-6baf-495f-a2fc-2495633716fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:29.980659 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-70bd8e77-6baf-495f-a2fc-2495633716fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:29.982124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-70bd8e77-6baf-495f-a2fc-2495633716fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:29.992580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-70bd8e77-6baf-495f-a2fc-2495633716fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1743 microversion: 2.20 time: 0.679223 Sep 08 13:56:29.997707 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 161/315] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:29 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1743 bytes in 685 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:30.019896 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-5d44a35c-6af9-4a40-9abb-9ef5b93df520 req-d5378894-1e6c-487c-9d66-f13ecf8f007e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c14f6817-e3ab-4a36-ba23-590498bf1b25", "name": "network-vif-plugged", "status": "completed", "tag": "4894f431-b6f6-46b3-95db-e9eb88227e33"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:56:30.043283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-5d44a35c-6af9-4a40-9abb-9ef5b93df520 req-d5378894-1e6c-487c-9d66-f13ecf8f007e service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:30.053807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-5d44a35c-6af9-4a40-9abb-9ef5b93df520 req-d5378894-1e6c-487c-9d66-f13ecf8f007e service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:30.055704 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-5d44a35c-6af9-4a40-9abb-9ef5b93df520 req-d5378894-1e6c-487c-9d66-f13ecf8f007e service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:30.090495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-5d44a35c-6af9-4a40-9abb-9ef5b93df520 req-d5378894-1e6c-487c-9d66-f13ecf8f007e service nova] Creating event network-vif-plugged:4894f431-b6f6-46b3-95db-e9eb88227e33 for instance c14f6817-e3ab-4a36-ba23-590498bf1b25 on hv-cinder-85646 Sep 08 13:56:30.111674 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-5d44a35c-6af9-4a40-9abb-9ef5b93df520 req-d5378894-1e6c-487c-9d66-f13ecf8f007e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.112100 Sep 08 13:56:30.117337 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 162/316] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:56:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:56:30.541701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-a1a4c2ac-5d56-4dd0-bcc7-dd49a6beede7 req-80329554-a51c-402f-90a8-53594a87cdc3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "19db4aa2-f574-43fa-bbf8-c3042b243152", "tag": "2ebf3a50-af15-4ecc-820b-87ad0a9ff652"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:56:30.555898 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-a1a4c2ac-5d56-4dd0-bcc7-dd49a6beede7 req-80329554-a51c-402f-90a8-53594a87cdc3 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:30.562873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-a1a4c2ac-5d56-4dd0-bcc7-dd49a6beede7 req-80329554-a51c-402f-90a8-53594a87cdc3 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:30.562873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-a1a4c2ac-5d56-4dd0-bcc7-dd49a6beede7 req-80329554-a51c-402f-90a8-53594a87cdc3 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:30.570800 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-a1a4c2ac-5d56-4dd0-bcc7-dd49a6beede7 req-80329554-a51c-402f-90a8-53594a87cdc3 service nova] Creating event network-changed:2ebf3a50-af15-4ecc-820b-87ad0a9ff652 for instance 19db4aa2-f574-43fa-bbf8-c3042b243152 on hv-cinder-85646 Sep 08 13:56:30.575715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-a1a4c2ac-5d56-4dd0-bcc7-dd49a6beede7 req-80329554-a51c-402f-90a8-53594a87cdc3 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038721 Sep 08 13:56:30.576678 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 163/317] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:56:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:56:30.706387 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-877ca352-4c6d-4552-adc0-115b4afec82e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Flavor limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 13:56:30.706878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-877ca352-4c6d-4552-adc0-115b4afec82e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Image limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 13:56:30.707345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-877ca352-4c6d-4552-adc0-115b4afec82e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Flavor pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 13:56:30.707759 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-877ca352-4c6d-4552-adc0-115b4afec82e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Image pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 13:56:30.708176 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-877ca352-4c6d-4552-adc0-115b4afec82e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 13:56:30.726010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-877ca352-4c6d-4552-adc0-115b4afec82e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:30.727277 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-877ca352-4c6d-4552-adc0-115b4afec82e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:30.727277 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-877ca352-4c6d-4552-adc0-115b4afec82e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:30.728022 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-877ca352-4c6d-4552-adc0-115b4afec82e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:30.728491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-877ca352-4c6d-4552-adc0-115b4afec82e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:30.728924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-877ca352-4c6d-4552-adc0-115b4afec82e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:30.769768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-877ca352-4c6d-4552-adc0-115b4afec82e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Getting quotas for project a4e0fba6cab34c429c73fcfac7ebfd94. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:56:30.776732 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-877ca352-4c6d-4552-adc0-115b4afec82e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Getting quotas for user eca89f435299415c8448931e73f5b0fc and project a4e0fba6cab34c429c73fcfac7ebfd94. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:56:30.802067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-877ca352-4c6d-4552-adc0-115b4afec82e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Going to run 1 instances... {{(pid=127601) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 13:56:30.814031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-877ca352-4c6d-4552-adc0-115b4afec82e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='20b30425-92e5-4d78-8502-45557526ea49',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127601) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 13:56:31.038794 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b6dc2fa8-d408-4990-a6bb-57a318c1e956 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:31.040030 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b6dc2fa8-d408-4990-a6bb-57a318c1e956 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:31.093141 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b6dc2fa8-d408-4990-a6bb-57a318c1e956 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:31.094326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b6dc2fa8-d408-4990-a6bb-57a318c1e956 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:31.095552 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b6dc2fa8-d408-4990-a6bb-57a318c1e956 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:31.100182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b6dc2fa8-d408-4990-a6bb-57a318c1e956 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:31.102988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b6dc2fa8-d408-4990-a6bb-57a318c1e956 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:31.107364 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b6dc2fa8-d408-4990-a6bb-57a318c1e956 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:31.133132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-877ca352-4c6d-4552-adc0-115b4afec82e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 2.807408 Sep 08 13:56:31.140906 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 155/318] 10.222.0.25 () {60 vars in 1147 bytes} [Thu Sep 8 13:56:28 2022] POST /compute/v2.1/servers => generated 402 bytes in 2821 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 08 13:56:31.190367 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6d4bfd5a-8ee1-40d4-a854-05dda9db8c05 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:31.191998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6d4bfd5a-8ee1-40d4-a854-05dda9db8c05 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:31.253206 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:31.912455 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b6dc2fa8-d408-4990-a6bb-57a318c1e956 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:31.913830 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6d4bfd5a-8ee1-40d4-a854-05dda9db8c05 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:31.924155 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b6dc2fa8-d408-4990-a6bb-57a318c1e956 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:31.929842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b6dc2fa8-d408-4990-a6bb-57a318c1e956 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:31.932890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b6dc2fa8-d408-4990-a6bb-57a318c1e956 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1738 microversion: 2.20 time: 0.909353 Sep 08 13:56:31.938434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 164/319] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:31 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1738 bytes in 913 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:31.987336 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6d4bfd5a-8ee1-40d4-a854-05dda9db8c05 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1239 microversion: 2.1 time: 0.835657 Sep 08 13:56:31.988558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 156/320] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:31 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1239 bytes in 838 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:32.989140 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d75df9e5-4480-4801-a113-656b8e7cf047 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:32.991225 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d75df9e5-4480-4801-a113-656b8e7cf047 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:33.008329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d75df9e5-4480-4801-a113-656b8e7cf047 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:33.008329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d75df9e5-4480-4801-a113-656b8e7cf047 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:33.008329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d75df9e5-4480-4801-a113-656b8e7cf047 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:33.008976 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d75df9e5-4480-4801-a113-656b8e7cf047 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:33.009677 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d75df9e5-4480-4801-a113-656b8e7cf047 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:33.010125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d75df9e5-4480-4801-a113-656b8e7cf047 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:33.257396 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8edc6cd1-db2e-4405-9a72-e6b6a907f608 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Action: 'create', calling method: .version_select at 0x7fce023b3c10>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-720914496", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBD1vR0PdJuUBKEBWtHiQRxEvv4PcW9BauEI2A1zv2O4fhm8o/mx4FbAcPQmeYFGZsnrRnNk3h7pWeNdMMenu9jjXM8Ahrrqq0DHufiWpxnb2KWBk//Oh4R7wLRP3I5/C5Q=="}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:56:33.288733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-8edc6cd1-db2e-4405-9a72-e6b6a907f608 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Getting quotas for project 45579d93010d4ed0b80cad3a8b85ea50. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:56:33.303776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-8edc6cd1-db2e-4405-9a72-e6b6a907f608 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Getting quotas for user 772205d93ce047e9ad4ddefadec300f4 and project 45579d93010d4ed0b80cad3a8b85ea50. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:56:33.440259 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8edc6cd1-db2e-4405-9a72-e6b6a907f608 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 399 microversion: 2.1 time: 0.659036 Sep 08 13:56:33.441162 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 165/321] 10.222.0.25 () {60 vars in 1159 bytes} [Thu Sep 8 13:56:32 2022] POST /compute/v2.1/os-keypairs => generated 399 bytes in 661 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:56:33.454898 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b9b333f3-ed4b-465f-838a-f69044085f3f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:33.454898 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b9b333f3-ed4b-465f-838a-f69044085f3f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:33.502771 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b9b333f3-ed4b-465f-838a-f69044085f3f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:33.512240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b9b333f3-ed4b-465f-838a-f69044085f3f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:33.512240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b9b333f3-ed4b-465f-838a-f69044085f3f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:33.514170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b9b333f3-ed4b-465f-838a-f69044085f3f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:33.518049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b9b333f3-ed4b-465f-838a-f69044085f3f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:33.524928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b9b333f3-ed4b-465f-838a-f69044085f3f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:33.752770 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d75df9e5-4480-4801-a113-656b8e7cf047 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:33.765612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d75df9e5-4480-4801-a113-656b8e7cf047 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:33.767602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d75df9e5-4480-4801-a113-656b8e7cf047 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:33.771963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d75df9e5-4480-4801-a113-656b8e7cf047 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1738 microversion: 2.20 time: 0.783806 Sep 08 13:56:33.774583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 157/322] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:32 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1738 bytes in 788 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:56:33.829764 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b9b333f3-ed4b-465f-838a-f69044085f3f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:33.847631 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b9b333f3-ed4b-465f-838a-f69044085f3f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1243 microversion: 2.1 time: 0.398950 Sep 08 13:56:33.848909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 166/323] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:33 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1243 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:34.018082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0992e8dd-9566-4bb4-97bf-c152f1b08e51 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1745126936", "imageRef": "20b30425-92e5-4d78-8502-45557526ea49", "flavorRef": "452", "networks": [{"uuid": "2fd138c6-fd4e-45ba-b3b6-39ff66a4e89f"}]}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:56:34.178266 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.network.neutron [None req-0992e8dd-9566-4bb4-97bf-c152f1b08e51 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] validate_networks() for [('2fd138c6-fd4e-45ba-b3b6-39ff66a4e89f', None, None, None, None, None)] {{(pid=127601) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 13:56:35.072502 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-0992e8dd-9566-4bb4-97bf-c152f1b08e51 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Flavor limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 13:56:35.073988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-0992e8dd-9566-4bb4-97bf-c152f1b08e51 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Image limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 13:56:35.074841 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-0992e8dd-9566-4bb4-97bf-c152f1b08e51 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Flavor pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 13:56:35.076782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-0992e8dd-9566-4bb4-97bf-c152f1b08e51 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Image pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 13:56:35.077404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-0992e8dd-9566-4bb4-97bf-c152f1b08e51 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 13:56:35.092523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0992e8dd-9566-4bb4-97bf-c152f1b08e51 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:35.093214 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0992e8dd-9566-4bb4-97bf-c152f1b08e51 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:35.094732 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0992e8dd-9566-4bb4-97bf-c152f1b08e51 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:35.097775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0992e8dd-9566-4bb4-97bf-c152f1b08e51 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:35.097775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0992e8dd-9566-4bb4-97bf-c152f1b08e51 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:35.098314 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0992e8dd-9566-4bb4-97bf-c152f1b08e51 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:35.127380 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-0992e8dd-9566-4bb4-97bf-c152f1b08e51 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Getting quotas for project 1363660dde8f4e4295adc44c6ab87f8b. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:56:35.140859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-0992e8dd-9566-4bb4-97bf-c152f1b08e51 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Getting quotas for user 5ae9377cf36e4086aa38c07b021fdead and project 1363660dde8f4e4295adc44c6ab87f8b. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:56:35.172192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0992e8dd-9566-4bb4-97bf-c152f1b08e51 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Going to run 1 instances... {{(pid=127601) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 13:56:35.174498 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b7528902-8c1a-41a5-8027-0d0dd2385c9f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Action: 'create', calling method: .version_select at 0x7fce030bef70>, body: {"keypair": {"name": "tempest-TestShelveInstance-1782896132", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNeA4Ja421D3eHZHMlO9RxqrjIbEDut0iGWzIAYG0b6jBeLkRlp1z9vf+GIzN8YdmLBOnzCe4LxDeG/Ib8rX/ZLCymWwIao19+QJJCDX1Qni/KQmkK840yrQUD8zy98aeQ=="}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:56:35.175030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0992e8dd-9566-4bb4-97bf-c152f1b08e51 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='20b30425-92e5-4d78-8502-45557526ea49',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127601) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 13:56:35.196503 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-b7528902-8c1a-41a5-8027-0d0dd2385c9f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Getting quotas for project 5be47c3b2c4c4fc2a1db117433cbcfc4. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:56:35.203802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-b7528902-8c1a-41a5-8027-0d0dd2385c9f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Getting quotas for user f2026170f4674b63a37534c14689bd10 and project 5be47c3b2c4c4fc2a1db117433cbcfc4. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:56:35.324352 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b7528902-8c1a-41a5-8027-0d0dd2385c9f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.569438 Sep 08 13:56:35.330756 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 167/324] 10.222.0.25 () {60 vars in 1159 bytes} [Thu Sep 8 13:56:34 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 576 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:56:35.342169 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ffc7d4f5-d549-41a8-a519-de81cee191d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:35.345696 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ffc7d4f5-d549-41a8-a519-de81cee191d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:35.363180 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ffc7d4f5-d549-41a8-a519-de81cee191d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:35.364229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ffc7d4f5-d549-41a8-a519-de81cee191d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:35.365720 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ffc7d4f5-d549-41a8-a519-de81cee191d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:35.372243 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0992e8dd-9566-4bb4-97bf-c152f1b08e51 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.354299 Sep 08 13:56:35.375093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 158/325] 10.222.0.25 () {60 vars in 1147 bytes} [Thu Sep 8 13:56:34 2022] POST /compute/v2.1/servers => generated 380 bytes in 1361 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 08 13:56:35.381758 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ffc7d4f5-d549-41a8-a519-de81cee191d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:35.383500 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ffc7d4f5-d549-41a8-a519-de81cee191d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:35.383976 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ffc7d4f5-d549-41a8-a519-de81cee191d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:35.394503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1213b735-ddfb-44c9-af64-d18447831b97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:35.398776 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1213b735-ddfb-44c9-af64-d18447831b97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:35.419470 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1213b735-ddfb-44c9-af64-d18447831b97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:35.420450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1213b735-ddfb-44c9-af64-d18447831b97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:35.421164 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1213b735-ddfb-44c9-af64-d18447831b97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:35.422128 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1213b735-ddfb-44c9-af64-d18447831b97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:35.422841 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1213b735-ddfb-44c9-af64-d18447831b97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:35.423663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1213b735-ddfb-44c9-af64-d18447831b97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:35.571502 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1213b735-ddfb-44c9-af64-d18447831b97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:35.597128 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1213b735-ddfb-44c9-af64-d18447831b97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1291 microversion: 2.1 time: 0.220607 Sep 08 13:56:35.598345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 159/326] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:35 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1291 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:35.605527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7a139b4e-44cb-41c2-86e2-e140527b17f1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:35.606562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7a139b4e-44cb-41c2-86e2-e140527b17f1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:35.807928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7a139b4e-44cb-41c2-86e2-e140527b17f1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:35.807928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7a139b4e-44cb-41c2-86e2-e140527b17f1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1219 microversion: 2.1 time: 0.188687 Sep 08 13:56:35.807928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 160/327] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:35 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1219 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:35.809838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ffc7d4f5-d549-41a8-a519-de81cee191d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:35.840911 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ffc7d4f5-d549-41a8-a519-de81cee191d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:35.842378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ffc7d4f5-d549-41a8-a519-de81cee191d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:35.843892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ffc7d4f5-d549-41a8-a519-de81cee191d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1738 microversion: 2.20 time: 0.514977 Sep 08 13:56:35.844089 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 168/328] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:35 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1738 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:36.628933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-31ecf0cc-c2d2-42a2-a81a-efb8187ce86d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:36.630000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-31ecf0cc-c2d2-42a2-a81a-efb8187ce86d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:36.664157 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31ecf0cc-c2d2-42a2-a81a-efb8187ce86d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:36.664703 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31ecf0cc-c2d2-42a2-a81a-efb8187ce86d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:36.665156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31ecf0cc-c2d2-42a2-a81a-efb8187ce86d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:36.665913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31ecf0cc-c2d2-42a2-a81a-efb8187ce86d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:36.669154 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31ecf0cc-c2d2-42a2-a81a-efb8187ce86d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:36.671759 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31ecf0cc-c2d2-42a2-a81a-efb8187ce86d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:36.832079 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-449a66c9-98cd-4f44-8cf3-b18faf8fb091 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:36.832079 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-449a66c9-98cd-4f44-8cf3-b18faf8fb091 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:36.840932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-31ecf0cc-c2d2-42a2-a81a-efb8187ce86d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:36.854033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-31ecf0cc-c2d2-42a2-a81a-efb8187ce86d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1299 microversion: 2.1 time: 0.233242 Sep 08 13:56:36.855433 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 161/329] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:36 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1299 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:36.876930 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-78372c92-b5f4-460c-8844-f86bfa7d457e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:36.877550 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-78372c92-b5f4-460c-8844-f86bfa7d457e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:36.893212 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-78372c92-b5f4-460c-8844-f86bfa7d457e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:36.894199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-78372c92-b5f4-460c-8844-f86bfa7d457e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:36.901183 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-78372c92-b5f4-460c-8844-f86bfa7d457e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:36.909824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-78372c92-b5f4-460c-8844-f86bfa7d457e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:36.909824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-78372c92-b5f4-460c-8844-f86bfa7d457e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:36.909824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-78372c92-b5f4-460c-8844-f86bfa7d457e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:36.912531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-449a66c9-98cd-4f44-8cf3-b18faf8fb091 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:36.921744 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-449a66c9-98cd-4f44-8cf3-b18faf8fb091 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1219 microversion: 2.1 time: 0.101851 Sep 08 13:56:36.927868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 169/330] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:36 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1219 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:37.350923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-78372c92-b5f4-460c-8844-f86bfa7d457e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:37.357572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-78372c92-b5f4-460c-8844-f86bfa7d457e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:37.362216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-78372c92-b5f4-460c-8844-f86bfa7d457e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:37.370292 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-78372c92-b5f4-460c-8844-f86bfa7d457e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1738 microversion: 2.20 time: 0.496554 Sep 08 13:56:37.370619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 162/331] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:36 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1738 bytes in 498 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:37.878655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6d44eb22-2a8c-4595-a160-5327b7001eb1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:37.880089 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6d44eb22-2a8c-4595-a160-5327b7001eb1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:37.910276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d44eb22-2a8c-4595-a160-5327b7001eb1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:37.911298 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d44eb22-2a8c-4595-a160-5327b7001eb1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:37.912605 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d44eb22-2a8c-4595-a160-5327b7001eb1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:37.915965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d44eb22-2a8c-4595-a160-5327b7001eb1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:37.917606 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d44eb22-2a8c-4595-a160-5327b7001eb1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:37.920305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d44eb22-2a8c-4595-a160-5327b7001eb1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:37.945078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-16768ec4-da5c-495f-9aa2-e70d5febb4dd tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:37.946564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-16768ec4-da5c-495f-9aa2-e70d5febb4dd tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:38.135228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-16768ec4-da5c-495f-9aa2-e70d5febb4dd tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:38.149218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-16768ec4-da5c-495f-9aa2-e70d5febb4dd tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1219 microversion: 2.1 time: 0.207127 Sep 08 13:56:38.150979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 163/332] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:37 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1219 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:56:38.400705 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-048cac0d-68f5-4193-8665-8436dc85a394 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:38.401540 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-048cac0d-68f5-4193-8665-8436dc85a394 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:38.427573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-048cac0d-68f5-4193-8665-8436dc85a394 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:38.429156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-048cac0d-68f5-4193-8665-8436dc85a394 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:38.431178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-048cac0d-68f5-4193-8665-8436dc85a394 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:38.435814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-048cac0d-68f5-4193-8665-8436dc85a394 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:38.435814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-048cac0d-68f5-4193-8665-8436dc85a394 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:38.435814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-048cac0d-68f5-4193-8665-8436dc85a394 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:38.752206 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6d44eb22-2a8c-4595-a160-5327b7001eb1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:38.761802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6d44eb22-2a8c-4595-a160-5327b7001eb1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1365 microversion: 2.1 time: 0.888033 Sep 08 13:56:38.791992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 170/333] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:37 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1365 bytes in 916 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:38.882197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-048cac0d-68f5-4193-8665-8436dc85a394 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:38.885070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:38.895279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-048cac0d-68f5-4193-8665-8436dc85a394 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:38.896811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-048cac0d-68f5-4193-8665-8436dc85a394 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:38.898285 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-048cac0d-68f5-4193-8665-8436dc85a394 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1738 microversion: 2.20 time: 0.503876 Sep 08 13:56:38.899630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 164/334] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:38 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1738 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:56:39.177128 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-782ccfe2-574e-494c-aa25-35cf227382d2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:39.179272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-782ccfe2-574e-494c-aa25-35cf227382d2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:39.181212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=85,nova_api:INSERT=3 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:39.186405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-782ccfe2-574e-494c-aa25-35cf227382d2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:39.187415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-782ccfe2-574e-494c-aa25-35cf227382d2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:39.187415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-782ccfe2-574e-494c-aa25-35cf227382d2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:39.188044 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-782ccfe2-574e-494c-aa25-35cf227382d2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:39.188682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-782ccfe2-574e-494c-aa25-35cf227382d2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:39.189106 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-782ccfe2-574e-494c-aa25-35cf227382d2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:39.283096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-782ccfe2-574e-494c-aa25-35cf227382d2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:39.294073 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-782ccfe2-574e-494c-aa25-35cf227382d2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1215 microversion: 2.1 time: 0.120779 Sep 08 13:56:39.295275 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 171/335] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:39 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1215 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:39.800730 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fd667ac6-8844-49d9-aafa-27a26c70cf25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:39.802054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fd667ac6-8844-49d9-aafa-27a26c70cf25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:39.811388 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd667ac6-8844-49d9-aafa-27a26c70cf25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:39.814949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd667ac6-8844-49d9-aafa-27a26c70cf25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:39.814949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd667ac6-8844-49d9-aafa-27a26c70cf25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:39.814949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd667ac6-8844-49d9-aafa-27a26c70cf25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:39.814949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd667ac6-8844-49d9-aafa-27a26c70cf25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:39.817700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd667ac6-8844-49d9-aafa-27a26c70cf25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:39.934720 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-07a02f55-f711-4393-826e-55eeca30ddfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:39.935759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-07a02f55-f711-4393-826e-55eeca30ddfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:39.946950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-07a02f55-f711-4393-826e-55eeca30ddfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:39.947694 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-07a02f55-f711-4393-826e-55eeca30ddfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:39.948255 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-07a02f55-f711-4393-826e-55eeca30ddfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:39.949074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-07a02f55-f711-4393-826e-55eeca30ddfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:39.952449 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-07a02f55-f711-4393-826e-55eeca30ddfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:39.953263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-07a02f55-f711-4393-826e-55eeca30ddfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:40.225702 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-07a02f55-f711-4393-826e-55eeca30ddfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:40.239656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-07a02f55-f711-4393-826e-55eeca30ddfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:40.253417 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-07a02f55-f711-4393-826e-55eeca30ddfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:40.265477 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-07a02f55-f711-4393-826e-55eeca30ddfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1738 microversion: 2.20 time: 0.339396 Sep 08 13:56:40.281549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 172/336] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:39 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1738 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:40.307306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fd667ac6-8844-49d9-aafa-27a26c70cf25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:40.335510 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fd667ac6-8844-49d9-aafa-27a26c70cf25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1365 microversion: 2.1 time: 0.522565 Sep 08 13:56:40.335510 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 165/337] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:39 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1365 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:40.335716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-944a591d-d10d-4ad1-b0da-58002a4fc0d8 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:40.335716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-944a591d-d10d-4ad1-b0da-58002a4fc0d8 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:40.352717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-944a591d-d10d-4ad1-b0da-58002a4fc0d8 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:40.356151 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-944a591d-d10d-4ad1-b0da-58002a4fc0d8 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:40.360500 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-944a591d-d10d-4ad1-b0da-58002a4fc0d8 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:40.362005 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-944a591d-d10d-4ad1-b0da-58002a4fc0d8 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:40.366873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-944a591d-d10d-4ad1-b0da-58002a4fc0d8 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:40.372939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-944a591d-d10d-4ad1-b0da-58002a4fc0d8 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:40.517868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-944a591d-d10d-4ad1-b0da-58002a4fc0d8 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:40.533612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-944a591d-d10d-4ad1-b0da-58002a4fc0d8 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1215 microversion: 2.1 time: 0.213040 Sep 08 13:56:40.547110 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 173/338] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:40 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1215 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:56:41.301351 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-157ea731-0179-4e54-8950-83bd45936784 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:41.301930 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-157ea731-0179-4e54-8950-83bd45936784 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:41.347555 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-85ff0340-9d92-4984-8dae-8b4e091d95c5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:41.348757 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-85ff0340-9d92-4984-8dae-8b4e091d95c5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:41.352118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-157ea731-0179-4e54-8950-83bd45936784 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:41.353615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-157ea731-0179-4e54-8950-83bd45936784 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:41.354082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-157ea731-0179-4e54-8950-83bd45936784 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:41.360705 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-157ea731-0179-4e54-8950-83bd45936784 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:41.361228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-157ea731-0179-4e54-8950-83bd45936784 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:41.361813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-157ea731-0179-4e54-8950-83bd45936784 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:41.364799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85ff0340-9d92-4984-8dae-8b4e091d95c5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:41.367387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85ff0340-9d92-4984-8dae-8b4e091d95c5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:41.374920 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85ff0340-9d92-4984-8dae-8b4e091d95c5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:41.375964 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85ff0340-9d92-4984-8dae-8b4e091d95c5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:41.376501 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85ff0340-9d92-4984-8dae-8b4e091d95c5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:41.377057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85ff0340-9d92-4984-8dae-8b4e091d95c5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:41.761674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-157ea731-0179-4e54-8950-83bd45936784 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:41.779426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-157ea731-0179-4e54-8950-83bd45936784 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:41.780910 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-157ea731-0179-4e54-8950-83bd45936784 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:41.782129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-157ea731-0179-4e54-8950-83bd45936784 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1738 microversion: 2.20 time: 0.487060 Sep 08 13:56:41.783040 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 166/339] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:41 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1738 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:41.789875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-85c1c6cb-22d6-4a26-b46f-0f38c4835b75 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:41.790516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-85c1c6cb-22d6-4a26-b46f-0f38c4835b75 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:41.837422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85c1c6cb-22d6-4a26-b46f-0f38c4835b75 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:41.837852 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85c1c6cb-22d6-4a26-b46f-0f38c4835b75 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:41.840039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85c1c6cb-22d6-4a26-b46f-0f38c4835b75 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:41.841544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85c1c6cb-22d6-4a26-b46f-0f38c4835b75 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:41.842024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85c1c6cb-22d6-4a26-b46f-0f38c4835b75 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:41.843141 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85c1c6cb-22d6-4a26-b46f-0f38c4835b75 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:41.978871 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-85c1c6cb-22d6-4a26-b46f-0f38c4835b75 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:41.987364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-85c1c6cb-22d6-4a26-b46f-0f38c4835b75 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1271 microversion: 2.1 time: 0.203236 Sep 08 13:56:41.987960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 167/340] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:41 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1271 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:42.664939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-85ff0340-9d92-4984-8dae-8b4e091d95c5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:42.701521 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-85ff0340-9d92-4984-8dae-8b4e091d95c5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1365 microversion: 2.1 time: 1.345788 Sep 08 13:56:42.707527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 174/341] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:41 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1365 bytes in 1362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:56:42.812052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6c1ea29b-1aae-48b4-bbbc-b86a1e793c73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:42.813439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6c1ea29b-1aae-48b4-bbbc-b86a1e793c73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:42.822213 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c1ea29b-1aae-48b4-bbbc-b86a1e793c73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:42.823772 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c1ea29b-1aae-48b4-bbbc-b86a1e793c73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:42.827945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c1ea29b-1aae-48b4-bbbc-b86a1e793c73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:42.827945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c1ea29b-1aae-48b4-bbbc-b86a1e793c73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:42.827945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c1ea29b-1aae-48b4-bbbc-b86a1e793c73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:42.827945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c1ea29b-1aae-48b4-bbbc-b86a1e793c73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:42.999798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6c1ea29b-1aae-48b4-bbbc-b86a1e793c73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:43.016633 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d85fac41-dbb2-4e3e-bf39-dc4df304e027 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:43.017747 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d85fac41-dbb2-4e3e-bf39-dc4df304e027 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:43.034113 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d85fac41-dbb2-4e3e-bf39-dc4df304e027 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:43.034900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6c1ea29b-1aae-48b4-bbbc-b86a1e793c73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:43.036840 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d85fac41-dbb2-4e3e-bf39-dc4df304e027 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:43.037744 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d85fac41-dbb2-4e3e-bf39-dc4df304e027 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:43.039665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6c1ea29b-1aae-48b4-bbbc-b86a1e793c73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:43.042353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d85fac41-dbb2-4e3e-bf39-dc4df304e027 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:43.043718 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6c1ea29b-1aae-48b4-bbbc-b86a1e793c73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1738 microversion: 2.20 time: 0.235580 Sep 08 13:56:43.044517 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d85fac41-dbb2-4e3e-bf39-dc4df304e027 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:43.045346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d85fac41-dbb2-4e3e-bf39-dc4df304e027 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:43.049909 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 168/342] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:42 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1738 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:43.191726 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d85fac41-dbb2-4e3e-bf39-dc4df304e027 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:43.200626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d85fac41-dbb2-4e3e-bf39-dc4df304e027 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1271 microversion: 2.1 time: 0.190660 Sep 08 13:56:43.201603 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 175/343] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:43 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1271 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:43.733131 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ef71f519-01d3-4b1b-aa8d-ed60c3b67258 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:43.734086 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ef71f519-01d3-4b1b-aa8d-ed60c3b67258 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:43.757691 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef71f519-01d3-4b1b-aa8d-ed60c3b67258 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:43.758696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef71f519-01d3-4b1b-aa8d-ed60c3b67258 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:43.759361 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef71f519-01d3-4b1b-aa8d-ed60c3b67258 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:43.761354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef71f519-01d3-4b1b-aa8d-ed60c3b67258 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:43.762056 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef71f519-01d3-4b1b-aa8d-ed60c3b67258 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:43.764469 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef71f519-01d3-4b1b-aa8d-ed60c3b67258 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:43.983402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ef71f519-01d3-4b1b-aa8d-ed60c3b67258 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:43.995641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ef71f519-01d3-4b1b-aa8d-ed60c3b67258 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1365 microversion: 2.1 time: 0.268228 Sep 08 13:56:44.013918 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 169/344] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:43 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1365 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:44.081929 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-23658a7e-4c67-45e2-a80e-7f05ee41e5fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:44.083243 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-23658a7e-4c67-45e2-a80e-7f05ee41e5fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:44.106351 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-2d7a9292-2a64-41b2-b210-732e040cd742 req-2aafb09a-ad8a-48a8-8775-76798550451e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "82890cea-b89a-4020-b9aa-8a7d94ac6005", "tag": "c152a3b2-3cdc-424f-9020-7dba12d0cb18"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:56:44.118697 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-23658a7e-4c67-45e2-a80e-7f05ee41e5fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:44.119201 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-23658a7e-4c67-45e2-a80e-7f05ee41e5fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:44.131576 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-23658a7e-4c67-45e2-a80e-7f05ee41e5fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:44.135621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-23658a7e-4c67-45e2-a80e-7f05ee41e5fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:44.135621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-23658a7e-4c67-45e2-a80e-7f05ee41e5fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:44.135621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-23658a7e-4c67-45e2-a80e-7f05ee41e5fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:44.135932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-2d7a9292-2a64-41b2-b210-732e040cd742 req-2aafb09a-ad8a-48a8-8775-76798550451e service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:44.135932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-2d7a9292-2a64-41b2-b210-732e040cd742 req-2aafb09a-ad8a-48a8-8775-76798550451e service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:44.135932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-2d7a9292-2a64-41b2-b210-732e040cd742 req-2aafb09a-ad8a-48a8-8775-76798550451e service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:44.166241 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-2d7a9292-2a64-41b2-b210-732e040cd742 req-2aafb09a-ad8a-48a8-8775-76798550451e service nova] Creating event network-changed:c152a3b2-3cdc-424f-9020-7dba12d0cb18 for instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 on hv-cinder-85646 Sep 08 13:56:44.201472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-2d7a9292-2a64-41b2-b210-732e040cd742 req-2aafb09a-ad8a-48a8-8775-76798550451e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.090113 Sep 08 13:56:44.203354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 170/345] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:56:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:56:44.227724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6f03567b-1476-4db6-8172-e256c0fbcb3d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:44.229081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6f03567b-1476-4db6-8172-e256c0fbcb3d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:44.266420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f03567b-1476-4db6-8172-e256c0fbcb3d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:44.266420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f03567b-1476-4db6-8172-e256c0fbcb3d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:44.266420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f03567b-1476-4db6-8172-e256c0fbcb3d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:44.266420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f03567b-1476-4db6-8172-e256c0fbcb3d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:44.266947 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f03567b-1476-4db6-8172-e256c0fbcb3d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:44.266947 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f03567b-1476-4db6-8172-e256c0fbcb3d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:44.465990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6f03567b-1476-4db6-8172-e256c0fbcb3d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:44.490730 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6f03567b-1476-4db6-8172-e256c0fbcb3d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1277 microversion: 2.1 time: 0.253774 Sep 08 13:56:44.490730 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 171/346] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:44 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1277 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:44.743459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-23658a7e-4c67-45e2-a80e-7f05ee41e5fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:44.751659 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-23658a7e-4c67-45e2-a80e-7f05ee41e5fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:44.753118 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-23658a7e-4c67-45e2-a80e-7f05ee41e5fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:44.754317 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-23658a7e-4c67-45e2-a80e-7f05ee41e5fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1738 microversion: 2.20 time: 0.679335 Sep 08 13:56:44.754884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 176/347] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:44 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1738 bytes in 682 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:45.022873 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5ca31623-9e72-4f1f-b7f6-f5f92fd70097 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:45.023733 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5ca31623-9e72-4f1f-b7f6-f5f92fd70097 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:45.030341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5ca31623-9e72-4f1f-b7f6-f5f92fd70097 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:45.035102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5ca31623-9e72-4f1f-b7f6-f5f92fd70097 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:45.035102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5ca31623-9e72-4f1f-b7f6-f5f92fd70097 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:45.035102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5ca31623-9e72-4f1f-b7f6-f5f92fd70097 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:45.035102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5ca31623-9e72-4f1f-b7f6-f5f92fd70097 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:45.035514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5ca31623-9e72-4f1f-b7f6-f5f92fd70097 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:45.112713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=4 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:45.205487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5ca31623-9e72-4f1f-b7f6-f5f92fd70097 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:45.214894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5ca31623-9e72-4f1f-b7f6-f5f92fd70097 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1365 microversion: 2.1 time: 0.195689 Sep 08 13:56:45.216192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 172/348] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:45 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1365 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:45.505585 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9e8b4fa3-4bf8-4e26-916d-725cfb354f5e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:45.506403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9e8b4fa3-4bf8-4e26-916d-725cfb354f5e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:45.513975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e8b4fa3-4bf8-4e26-916d-725cfb354f5e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:45.514502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e8b4fa3-4bf8-4e26-916d-725cfb354f5e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:45.514938 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e8b4fa3-4bf8-4e26-916d-725cfb354f5e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:45.515657 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e8b4fa3-4bf8-4e26-916d-725cfb354f5e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:45.516305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e8b4fa3-4bf8-4e26-916d-725cfb354f5e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:45.516903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e8b4fa3-4bf8-4e26-916d-725cfb354f5e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:45.781283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-18df0378-91bb-4e5a-aac8-738ddaec4486 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:45.786236 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-18df0378-91bb-4e5a-aac8-738ddaec4486 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:45.798781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-18df0378-91bb-4e5a-aac8-738ddaec4486 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:45.799391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-18df0378-91bb-4e5a-aac8-738ddaec4486 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:45.799931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-18df0378-91bb-4e5a-aac8-738ddaec4486 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:45.800987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-18df0378-91bb-4e5a-aac8-738ddaec4486 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:45.801680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-18df0378-91bb-4e5a-aac8-738ddaec4486 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:45.802526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-18df0378-91bb-4e5a-aac8-738ddaec4486 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:46.069474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-18df0378-91bb-4e5a-aac8-738ddaec4486 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:46.096534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-18df0378-91bb-4e5a-aac8-738ddaec4486 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:46.099657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-18df0378-91bb-4e5a-aac8-738ddaec4486 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:46.103291 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-18df0378-91bb-4e5a-aac8-738ddaec4486 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1738 microversion: 2.20 time: 0.325287 Sep 08 13:56:46.104380 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 173/349] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:45 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1738 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:56:46.129684 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9e8b4fa3-4bf8-4e26-916d-725cfb354f5e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:46.153133 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9e8b4fa3-4bf8-4e26-916d-725cfb354f5e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1319 microversion: 2.1 time: 0.644783 Sep 08 13:56:46.153133 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 177/350] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:45 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1319 bytes in 647 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:46.239288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bb872710-e1d8-404f-a1c8-80d1af92ae47 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:46.240434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bb872710-e1d8-404f-a1c8-80d1af92ae47 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:46.260973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb872710-e1d8-404f-a1c8-80d1af92ae47 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:46.261613 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb872710-e1d8-404f-a1c8-80d1af92ae47 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:46.269359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb872710-e1d8-404f-a1c8-80d1af92ae47 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:46.271543 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb872710-e1d8-404f-a1c8-80d1af92ae47 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:46.272278 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb872710-e1d8-404f-a1c8-80d1af92ae47 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:46.272727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb872710-e1d8-404f-a1c8-80d1af92ae47 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:46.464231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bb872710-e1d8-404f-a1c8-80d1af92ae47 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:46.474373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bb872710-e1d8-404f-a1c8-80d1af92ae47 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1365 microversion: 2.1 time: 0.238723 Sep 08 13:56:46.475835 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 174/351] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:46 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1365 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:47.038105 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-32f83e40-4311-4e91-8d7b-c081f7f72284 req-4906409a-15ef-4bc9-ad8b-61f9e34edbcd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7ec70561-b436-4d03-bd60-6c5ae232b492", "tag": "238b9d36-b260-4554-b58e-6b5beab1736c"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:56:47.046006 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=14 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:47.049159 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-32f83e40-4311-4e91-8d7b-c081f7f72284 req-4906409a-15ef-4bc9-ad8b-61f9e34edbcd service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:47.049812 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-32f83e40-4311-4e91-8d7b-c081f7f72284 req-4906409a-15ef-4bc9-ad8b-61f9e34edbcd service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:47.050502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-32f83e40-4311-4e91-8d7b-c081f7f72284 req-4906409a-15ef-4bc9-ad8b-61f9e34edbcd service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:47.067715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-32f83e40-4311-4e91-8d7b-c081f7f72284 req-4906409a-15ef-4bc9-ad8b-61f9e34edbcd service nova] Creating event network-changed:238b9d36-b260-4554-b58e-6b5beab1736c for instance 7ec70561-b436-4d03-bd60-6c5ae232b492 on hv-cinder-85646 Sep 08 13:56:47.074174 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-32f83e40-4311-4e91-8d7b-c081f7f72284 req-4906409a-15ef-4bc9-ad8b-61f9e34edbcd service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044287 Sep 08 13:56:47.075339 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 178/352] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:56:47 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:56:47.133319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a7fb05a9-9a67-4e82-ad70-a0976e6141d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:47.133908 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a7fb05a9-9a67-4e82-ad70-a0976e6141d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:47.141212 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7fb05a9-9a67-4e82-ad70-a0976e6141d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:47.141919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7fb05a9-9a67-4e82-ad70-a0976e6141d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:47.142377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7fb05a9-9a67-4e82-ad70-a0976e6141d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:47.143285 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7fb05a9-9a67-4e82-ad70-a0976e6141d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:47.143952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7fb05a9-9a67-4e82-ad70-a0976e6141d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:47.144599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7fb05a9-9a67-4e82-ad70-a0976e6141d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:47.174324 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-80f93120-9305-424d-a663-58f9ba023256 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:47.175239 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-80f93120-9305-424d-a663-58f9ba023256 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:47.183854 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-80f93120-9305-424d-a663-58f9ba023256 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:47.184823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-80f93120-9305-424d-a663-58f9ba023256 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:47.185425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-80f93120-9305-424d-a663-58f9ba023256 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:47.186394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-80f93120-9305-424d-a663-58f9ba023256 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:47.187057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-80f93120-9305-424d-a663-58f9ba023256 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:47.187682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-80f93120-9305-424d-a663-58f9ba023256 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:47.358212 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a7fb05a9-9a67-4e82-ad70-a0976e6141d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:47.368774 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a7fb05a9-9a67-4e82-ad70-a0976e6141d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:47.373288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a7fb05a9-9a67-4e82-ad70-a0976e6141d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:47.374696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a7fb05a9-9a67-4e82-ad70-a0976e6141d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1738 microversion: 2.20 time: 0.246929 Sep 08 13:56:47.375807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 175/353] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:47 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1738 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:47.409780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-80f93120-9305-424d-a663-58f9ba023256 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:47.420808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-80f93120-9305-424d-a663-58f9ba023256 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1319 microversion: 2.1 time: 0.251040 Sep 08 13:56:47.436179 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 179/354] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:47 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1319 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:47.502043 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-71117c20-84f6-402e-a0c6-cc617cdad455 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:47.513630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-71117c20-84f6-402e-a0c6-cc617cdad455 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:47.513630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-71117c20-84f6-402e-a0c6-cc617cdad455 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:47.513630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-71117c20-84f6-402e-a0c6-cc617cdad455 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:47.513630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-71117c20-84f6-402e-a0c6-cc617cdad455 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:47.514056 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-71117c20-84f6-402e-a0c6-cc617cdad455 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:47.514056 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-71117c20-84f6-402e-a0c6-cc617cdad455 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:47.514056 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-71117c20-84f6-402e-a0c6-cc617cdad455 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:47.713903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-71117c20-84f6-402e-a0c6-cc617cdad455 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:47.724556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-71117c20-84f6-402e-a0c6-cc617cdad455 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1365 microversion: 2.1 time: 0.226740 Sep 08 13:56:47.725409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 176/355] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:47 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1365 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:48.408387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9c04b848-413c-4fc8-baae-0b2cb3969b2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:48.409772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9c04b848-413c-4fc8-baae-0b2cb3969b2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:48.424743 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9c04b848-413c-4fc8-baae-0b2cb3969b2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:48.426925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9c04b848-413c-4fc8-baae-0b2cb3969b2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:48.428642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9c04b848-413c-4fc8-baae-0b2cb3969b2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:48.432428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9c04b848-413c-4fc8-baae-0b2cb3969b2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:48.433346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9c04b848-413c-4fc8-baae-0b2cb3969b2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:48.434131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9c04b848-413c-4fc8-baae-0b2cb3969b2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:48.445555 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-77415f05-c593-4988-8c76-4ef6340178dc tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:48.446582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-77415f05-c593-4988-8c76-4ef6340178dc tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:48.449873 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=18 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:48.462307 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77415f05-c593-4988-8c76-4ef6340178dc tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:48.464902 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77415f05-c593-4988-8c76-4ef6340178dc tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:48.465778 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77415f05-c593-4988-8c76-4ef6340178dc tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:48.466770 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77415f05-c593-4988-8c76-4ef6340178dc tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:48.467452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77415f05-c593-4988-8c76-4ef6340178dc tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:48.468051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77415f05-c593-4988-8c76-4ef6340178dc tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:48.666983 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-77415f05-c593-4988-8c76-4ef6340178dc tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:48.686259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-77415f05-c593-4988-8c76-4ef6340178dc tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1319 microversion: 2.1 time: 0.245317 Sep 08 13:56:48.687613 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 177/356] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:48 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1319 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:56:48.750571 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a28cac81-83ae-497b-9af2-bd7d4d755d23 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:48.751657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a28cac81-83ae-497b-9af2-bd7d4d755d23 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:48.758500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a28cac81-83ae-497b-9af2-bd7d4d755d23 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:48.759065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a28cac81-83ae-497b-9af2-bd7d4d755d23 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:48.759440 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a28cac81-83ae-497b-9af2-bd7d4d755d23 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:48.760148 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a28cac81-83ae-497b-9af2-bd7d4d755d23 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:48.760954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a28cac81-83ae-497b-9af2-bd7d4d755d23 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:48.761387 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a28cac81-83ae-497b-9af2-bd7d4d755d23 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:48.762816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9c04b848-413c-4fc8-baae-0b2cb3969b2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:48.787815 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9c04b848-413c-4fc8-baae-0b2cb3969b2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:48.790206 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9c04b848-413c-4fc8-baae-0b2cb3969b2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:48.792039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9c04b848-413c-4fc8-baae-0b2cb3969b2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1738 microversion: 2.20 time: 0.387987 Sep 08 13:56:48.792731 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 180/357] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:48 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1738 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:48.930517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a28cac81-83ae-497b-9af2-bd7d4d755d23 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:48.938244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a28cac81-83ae-497b-9af2-bd7d4d755d23 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1365 microversion: 2.1 time: 0.193183 Sep 08 13:56:48.938826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 178/358] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:48 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1365 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:56:49.710921 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a139c2b8-13d6-4d01-9d97-425ebcaa74e3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:49.711507 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a139c2b8-13d6-4d01-9d97-425ebcaa74e3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:49.718507 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a139c2b8-13d6-4d01-9d97-425ebcaa74e3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:49.722832 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a139c2b8-13d6-4d01-9d97-425ebcaa74e3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:49.725013 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a139c2b8-13d6-4d01-9d97-425ebcaa74e3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:49.726423 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a139c2b8-13d6-4d01-9d97-425ebcaa74e3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:49.727280 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a139c2b8-13d6-4d01-9d97-425ebcaa74e3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:49.730179 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a139c2b8-13d6-4d01-9d97-425ebcaa74e3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:49.824506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-49190e6f-dd75-43ff-b5b8-e275b953a69e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:49.825786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-49190e6f-dd75-43ff-b5b8-e275b953a69e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:49.834385 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-49190e6f-dd75-43ff-b5b8-e275b953a69e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:49.837608 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-49190e6f-dd75-43ff-b5b8-e275b953a69e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:49.840841 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-49190e6f-dd75-43ff-b5b8-e275b953a69e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:49.841877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-49190e6f-dd75-43ff-b5b8-e275b953a69e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:49.842386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-49190e6f-dd75-43ff-b5b8-e275b953a69e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:49.842817 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-49190e6f-dd75-43ff-b5b8-e275b953a69e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:50.000142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a139c2b8-13d6-4d01-9d97-425ebcaa74e3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:50.017454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a139c2b8-13d6-4d01-9d97-425ebcaa74e3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1319 microversion: 2.1 time: 0.309119 Sep 08 13:56:50.017454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 181/359] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:49 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1319 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:50.047332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-49190e6f-dd75-43ff-b5b8-e275b953a69e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:50.051951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0eeeaccb-34ed-42eb-bc65-079830f62aeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:50.051951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0eeeaccb-34ed-42eb-bc65-079830f62aeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:50.058214 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-49190e6f-dd75-43ff-b5b8-e275b953a69e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:50.063118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-49190e6f-dd75-43ff-b5b8-e275b953a69e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:50.065265 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-49190e6f-dd75-43ff-b5b8-e275b953a69e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1738 microversion: 2.20 time: 0.244635 Sep 08 13:56:50.066032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 179/360] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:49 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1738 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:56:50.068256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0eeeaccb-34ed-42eb-bc65-079830f62aeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:50.068957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0eeeaccb-34ed-42eb-bc65-079830f62aeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:50.069430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0eeeaccb-34ed-42eb-bc65-079830f62aeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:50.070173 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0eeeaccb-34ed-42eb-bc65-079830f62aeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:50.070628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0eeeaccb-34ed-42eb-bc65-079830f62aeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:50.071241 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0eeeaccb-34ed-42eb-bc65-079830f62aeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:50.274954 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0eeeaccb-34ed-42eb-bc65-079830f62aeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:50.290056 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0eeeaccb-34ed-42eb-bc65-079830f62aeb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1365 microversion: 2.1 time: 0.272171 Sep 08 13:56:50.291365 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 182/361] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:50 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1365 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:51.040827 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ec54bacf-3e2b-4174-9761-0918a167ff14 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:51.043378 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ec54bacf-3e2b-4174-9761-0918a167ff14 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:51.048455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec54bacf-3e2b-4174-9761-0918a167ff14 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:51.048948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec54bacf-3e2b-4174-9761-0918a167ff14 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:51.049386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec54bacf-3e2b-4174-9761-0918a167ff14 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:51.050256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec54bacf-3e2b-4174-9761-0918a167ff14 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:51.050737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec54bacf-3e2b-4174-9761-0918a167ff14 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:51.051148 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec54bacf-3e2b-4174-9761-0918a167ff14 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:51.090045 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-84551abc-ff55-4e97-9d2d-0213be76d247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:51.090655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-84551abc-ff55-4e97-9d2d-0213be76d247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:51.097116 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84551abc-ff55-4e97-9d2d-0213be76d247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:51.097372 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84551abc-ff55-4e97-9d2d-0213be76d247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:51.097725 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84551abc-ff55-4e97-9d2d-0213be76d247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:51.098465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84551abc-ff55-4e97-9d2d-0213be76d247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:51.098921 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84551abc-ff55-4e97-9d2d-0213be76d247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:51.099348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84551abc-ff55-4e97-9d2d-0213be76d247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:51.253612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ec54bacf-3e2b-4174-9761-0918a167ff14 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:51.262549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ec54bacf-3e2b-4174-9761-0918a167ff14 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1319 microversion: 2.1 time: 0.226012 Sep 08 13:56:51.274485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-84551abc-ff55-4e97-9d2d-0213be76d247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:51.275586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 180/362] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:51 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1319 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:51.281877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-84551abc-ff55-4e97-9d2d-0213be76d247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:51.283511 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-84551abc-ff55-4e97-9d2d-0213be76d247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:51.284677 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-84551abc-ff55-4e97-9d2d-0213be76d247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1738 microversion: 2.20 time: 0.198659 Sep 08 13:56:51.285214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 183/363] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:51 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1738 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:56:51.327327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0adf3c93-1693-4979-8db5-9c21d481f8f2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:51.327852 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0adf3c93-1693-4979-8db5-9c21d481f8f2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:51.329751 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:51.335282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0adf3c93-1693-4979-8db5-9c21d481f8f2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:51.337494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0adf3c93-1693-4979-8db5-9c21d481f8f2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:51.337494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0adf3c93-1693-4979-8db5-9c21d481f8f2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:51.337678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0adf3c93-1693-4979-8db5-9c21d481f8f2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:51.338054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0adf3c93-1693-4979-8db5-9c21d481f8f2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:51.338507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0adf3c93-1693-4979-8db5-9c21d481f8f2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:51.720569 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0adf3c93-1693-4979-8db5-9c21d481f8f2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:51.728574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0adf3c93-1693-4979-8db5-9c21d481f8f2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1365 microversion: 2.1 time: 0.405458 Sep 08 13:56:51.729121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 181/364] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:51 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1365 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:52.286095 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ff58b45f-14b1-4618-b0d4-f4728177bce5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:52.293114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ff58b45f-14b1-4618-b0d4-f4728177bce5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:52.293616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff58b45f-14b1-4618-b0d4-f4728177bce5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:52.294106 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff58b45f-14b1-4618-b0d4-f4728177bce5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:52.294561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff58b45f-14b1-4618-b0d4-f4728177bce5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:52.295256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff58b45f-14b1-4618-b0d4-f4728177bce5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:52.295711 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff58b45f-14b1-4618-b0d4-f4728177bce5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:52.296134 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff58b45f-14b1-4618-b0d4-f4728177bce5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:52.312749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9a3c300e-b85c-455a-8b48-476b574e1569 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:52.317330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9a3c300e-b85c-455a-8b48-476b574e1569 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:52.324754 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a3c300e-b85c-455a-8b48-476b574e1569 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:52.333332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a3c300e-b85c-455a-8b48-476b574e1569 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:52.333332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a3c300e-b85c-455a-8b48-476b574e1569 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:52.333332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a3c300e-b85c-455a-8b48-476b574e1569 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:52.333332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a3c300e-b85c-455a-8b48-476b574e1569 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:52.333865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a3c300e-b85c-455a-8b48-476b574e1569 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:52.333865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:52.507262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ff58b45f-14b1-4618-b0d4-f4728177bce5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:52.510201 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:52.517966 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ff58b45f-14b1-4618-b0d4-f4728177bce5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1319 microversion: 2.1 time: 0.235652 Sep 08 13:56:52.518671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 184/365] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:52 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1319 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:52.531908 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9a3c300e-b85c-455a-8b48-476b574e1569 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:52.547625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9a3c300e-b85c-455a-8b48-476b574e1569 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:52.549030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9a3c300e-b85c-455a-8b48-476b574e1569 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:52.551181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9a3c300e-b85c-455a-8b48-476b574e1569 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1738 microversion: 2.20 time: 0.242945 Sep 08 13:56:52.562034 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 182/366] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:52 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1738 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:56:52.753432 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-60b7dfbd-fae9-43a2-bdc5-f93d7298a1b2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:52.754043 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-60b7dfbd-fae9-43a2-bdc5-f93d7298a1b2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:52.760555 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-60b7dfbd-fae9-43a2-bdc5-f93d7298a1b2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:52.761045 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-60b7dfbd-fae9-43a2-bdc5-f93d7298a1b2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:52.761483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-60b7dfbd-fae9-43a2-bdc5-f93d7298a1b2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:52.762207 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-60b7dfbd-fae9-43a2-bdc5-f93d7298a1b2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:52.762829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-60b7dfbd-fae9-43a2-bdc5-f93d7298a1b2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:52.763260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-60b7dfbd-fae9-43a2-bdc5-f93d7298a1b2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:52.923004 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-60b7dfbd-fae9-43a2-bdc5-f93d7298a1b2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:52.932052 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-60b7dfbd-fae9-43a2-bdc5-f93d7298a1b2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1365 microversion: 2.1 time: 0.182847 Sep 08 13:56:52.932716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 185/367] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:52 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1365 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:53.542524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f82eb680-0c84-4bf4-9489-e4deacdbb79d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:53.543109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f82eb680-0c84-4bf4-9489-e4deacdbb79d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:53.549565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f82eb680-0c84-4bf4-9489-e4deacdbb79d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:53.554786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f82eb680-0c84-4bf4-9489-e4deacdbb79d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:53.554786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f82eb680-0c84-4bf4-9489-e4deacdbb79d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:53.554786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f82eb680-0c84-4bf4-9489-e4deacdbb79d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:53.554786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f82eb680-0c84-4bf4-9489-e4deacdbb79d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:53.555177 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f82eb680-0c84-4bf4-9489-e4deacdbb79d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:53.579114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-684541a8-1c61-425b-a35d-f453c0b60ada tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:53.580443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-684541a8-1c61-425b-a35d-f453c0b60ada tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:53.587065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-684541a8-1c61-425b-a35d-f453c0b60ada tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:53.587560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-684541a8-1c61-425b-a35d-f453c0b60ada tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:53.587996 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-684541a8-1c61-425b-a35d-f453c0b60ada tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:53.588899 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-684541a8-1c61-425b-a35d-f453c0b60ada tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:53.589757 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-684541a8-1c61-425b-a35d-f453c0b60ada tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:53.590112 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-684541a8-1c61-425b-a35d-f453c0b60ada tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:53.724776 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f82eb680-0c84-4bf4-9489-e4deacdbb79d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:53.733433 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f82eb680-0c84-4bf4-9489-e4deacdbb79d tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1319 microversion: 2.1 time: 0.196153 Sep 08 13:56:53.734236 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 183/368] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:53 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1319 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:53.741328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-684541a8-1c61-425b-a35d-f453c0b60ada tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:53.749637 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-684541a8-1c61-425b-a35d-f453c0b60ada tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:53.751223 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-684541a8-1c61-425b-a35d-f453c0b60ada tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:53.752290 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-684541a8-1c61-425b-a35d-f453c0b60ada tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1738 microversion: 2.20 time: 0.177798 Sep 08 13:56:53.752847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 186/369] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:53 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1738 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:53.956086 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-631a19ac-f2b5-423c-8060-85cf86d36c6d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:53.956705 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-631a19ac-f2b5-423c-8060-85cf86d36c6d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:53.964794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-631a19ac-f2b5-423c-8060-85cf86d36c6d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:53.965946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-631a19ac-f2b5-423c-8060-85cf86d36c6d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:53.967048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-631a19ac-f2b5-423c-8060-85cf86d36c6d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:53.968070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-631a19ac-f2b5-423c-8060-85cf86d36c6d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:53.968778 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-631a19ac-f2b5-423c-8060-85cf86d36c6d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:53.969403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-631a19ac-f2b5-423c-8060-85cf86d36c6d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:54.142695 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-631a19ac-f2b5-423c-8060-85cf86d36c6d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:54.152686 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-631a19ac-f2b5-423c-8060-85cf86d36c6d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1365 microversion: 2.1 time: 0.200936 Sep 08 13:56:54.153275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 184/370] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:53 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1365 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:56:54.757522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6c60bd90-82be-4c82-a7c6-35fcf968f6a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:54.758123 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6c60bd90-82be-4c82-a7c6-35fcf968f6a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:54.765403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c60bd90-82be-4c82-a7c6-35fcf968f6a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:54.765927 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c60bd90-82be-4c82-a7c6-35fcf968f6a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:54.766637 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c60bd90-82be-4c82-a7c6-35fcf968f6a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:54.767971 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c60bd90-82be-4c82-a7c6-35fcf968f6a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:54.768491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c60bd90-82be-4c82-a7c6-35fcf968f6a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:54.768993 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c60bd90-82be-4c82-a7c6-35fcf968f6a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:54.782815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f5c488c9-99bf-4066-92fe-5bbd01002eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:54.784271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f5c488c9-99bf-4066-92fe-5bbd01002eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:54.798698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f5c488c9-99bf-4066-92fe-5bbd01002eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:54.799859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f5c488c9-99bf-4066-92fe-5bbd01002eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:54.800736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f5c488c9-99bf-4066-92fe-5bbd01002eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:54.810635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f5c488c9-99bf-4066-92fe-5bbd01002eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:54.812650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f5c488c9-99bf-4066-92fe-5bbd01002eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:54.813549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f5c488c9-99bf-4066-92fe-5bbd01002eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:55.095646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f5c488c9-99bf-4066-92fe-5bbd01002eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:55.110123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f5c488c9-99bf-4066-92fe-5bbd01002eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:55.111948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f5c488c9-99bf-4066-92fe-5bbd01002eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:55.113465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f5c488c9-99bf-4066-92fe-5bbd01002eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1720 microversion: 2.20 time: 0.340227 Sep 08 13:56:55.114373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 185/371] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:54 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1720 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:55.120544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-a5bb42bf-6022-42ba-ae6a-50a0c94579a9 req-851e098f-bd0c-49ed-b10f-9111394dd902 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "82890cea-b89a-4020-b9aa-8a7d94ac6005", "name": "network-vif-plugged", "status": "completed", "tag": "c152a3b2-3cdc-424f-9020-7dba12d0cb18"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:56:55.142326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-a5bb42bf-6022-42ba-ae6a-50a0c94579a9 req-851e098f-bd0c-49ed-b10f-9111394dd902 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:55.143169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-a5bb42bf-6022-42ba-ae6a-50a0c94579a9 req-851e098f-bd0c-49ed-b10f-9111394dd902 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:55.143842 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-a5bb42bf-6022-42ba-ae6a-50a0c94579a9 req-851e098f-bd0c-49ed-b10f-9111394dd902 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:55.165123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-a5bb42bf-6022-42ba-ae6a-50a0c94579a9 req-851e098f-bd0c-49ed-b10f-9111394dd902 service nova] Creating event network-vif-plugged:c152a3b2-3cdc-424f-9020-7dba12d0cb18 for instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 on hv-cinder-85646 Sep 08 13:56:55.179832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-a5bb42bf-6022-42ba-ae6a-50a0c94579a9 req-851e098f-bd0c-49ed-b10f-9111394dd902 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.062966 Sep 08 13:56:55.181182 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 186/372] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:56:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:56:55.188261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6b9d4b29-cf12-4ebf-98a4-3c3829d147c6 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:55.189771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6b9d4b29-cf12-4ebf-98a4-3c3829d147c6 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:55.200680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b9d4b29-cf12-4ebf-98a4-3c3829d147c6 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:55.202091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b9d4b29-cf12-4ebf-98a4-3c3829d147c6 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:55.203472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b9d4b29-cf12-4ebf-98a4-3c3829d147c6 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:55.203768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b9d4b29-cf12-4ebf-98a4-3c3829d147c6 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:55.204807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b9d4b29-cf12-4ebf-98a4-3c3829d147c6 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:55.205612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b9d4b29-cf12-4ebf-98a4-3c3829d147c6 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:55.256989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6c60bd90-82be-4c82-a7c6-35fcf968f6a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:55.266599 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6c60bd90-82be-4c82-a7c6-35fcf968f6a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1319 microversion: 2.1 time: 0.513651 Sep 08 13:56:55.282592 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 187/373] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:54 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1319 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:55.390275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6b9d4b29-cf12-4ebf-98a4-3c3829d147c6 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:55.400209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6b9d4b29-cf12-4ebf-98a4-3c3829d147c6 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1365 microversion: 2.1 time: 0.217598 Sep 08 13:56:55.401456 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 187/374] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:55 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1365 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:56.150749 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ed3651de-bd0d-48a6-ab67-c2aa29759758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:56.152849 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ed3651de-bd0d-48a6-ab67-c2aa29759758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:56.164283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed3651de-bd0d-48a6-ab67-c2aa29759758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:56.165261 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed3651de-bd0d-48a6-ab67-c2aa29759758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:56.166125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed3651de-bd0d-48a6-ab67-c2aa29759758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:56.167700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed3651de-bd0d-48a6-ab67-c2aa29759758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:56.168400 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed3651de-bd0d-48a6-ab67-c2aa29759758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:56.169092 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed3651de-bd0d-48a6-ab67-c2aa29759758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:56.298970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7009d90e-77a5-4285-86b6-9815798b8c5c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:56.299933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7009d90e-77a5-4285-86b6-9815798b8c5c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:56.308316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7009d90e-77a5-4285-86b6-9815798b8c5c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:56.309127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7009d90e-77a5-4285-86b6-9815798b8c5c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:56.309773 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7009d90e-77a5-4285-86b6-9815798b8c5c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:56.310563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7009d90e-77a5-4285-86b6-9815798b8c5c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:56.311056 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7009d90e-77a5-4285-86b6-9815798b8c5c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:56.311472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7009d90e-77a5-4285-86b6-9815798b8c5c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:56.508893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7009d90e-77a5-4285-86b6-9815798b8c5c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:56.518461 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7009d90e-77a5-4285-86b6-9815798b8c5c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1319 microversion: 2.1 time: 0.229354 Sep 08 13:56:56.519591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 188/375] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:56 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1319 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:56:56.524925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c682e089-b6e2-44a8-a94d-7b0b48ec5e02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:56.526404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c682e089-b6e2-44a8-a94d-7b0b48ec5e02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:56.540266 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c682e089-b6e2-44a8-a94d-7b0b48ec5e02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:56.540769 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c682e089-b6e2-44a8-a94d-7b0b48ec5e02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:56.541211 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c682e089-b6e2-44a8-a94d-7b0b48ec5e02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:56.541921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c682e089-b6e2-44a8-a94d-7b0b48ec5e02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:56.542414 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c682e089-b6e2-44a8-a94d-7b0b48ec5e02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:56.542839 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c682e089-b6e2-44a8-a94d-7b0b48ec5e02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:56.784049 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c682e089-b6e2-44a8-a94d-7b0b48ec5e02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:56.819455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c682e089-b6e2-44a8-a94d-7b0b48ec5e02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1365 microversion: 2.1 time: 0.298277 Sep 08 13:56:56.820338 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 189/376] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:56 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1365 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:56.968960 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ed3651de-bd0d-48a6-ab67-c2aa29759758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:56.985833 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ed3651de-bd0d-48a6-ab67-c2aa29759758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:56.987331 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ed3651de-bd0d-48a6-ab67-c2aa29759758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:56.988519 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ed3651de-bd0d-48a6-ab67-c2aa29759758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1720 microversion: 2.20 time: 0.847165 Sep 08 13:56:56.989316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 188/377] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:56 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1720 bytes in 849 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:57.548215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7d37a9d1-36db-4dcc-8bca-a19314387f8b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:57.549053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7d37a9d1-36db-4dcc-8bca-a19314387f8b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:57.556792 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7d37a9d1-36db-4dcc-8bca-a19314387f8b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:57.557578 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7d37a9d1-36db-4dcc-8bca-a19314387f8b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:57.558319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7d37a9d1-36db-4dcc-8bca-a19314387f8b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:57.559280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7d37a9d1-36db-4dcc-8bca-a19314387f8b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:57.559795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7d37a9d1-36db-4dcc-8bca-a19314387f8b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:57.560229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7d37a9d1-36db-4dcc-8bca-a19314387f8b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:57.750964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7d37a9d1-36db-4dcc-8bca-a19314387f8b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:57.762730 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7d37a9d1-36db-4dcc-8bca-a19314387f8b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1319 microversion: 2.1 time: 0.219278 Sep 08 13:56:57.763868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 190/378] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:57 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1319 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:56:57.843349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-811f5391-6b31-4746-b2c3-d3696a3d0f2f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:57.844406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-811f5391-6b31-4746-b2c3-d3696a3d0f2f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:57.853079 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-811f5391-6b31-4746-b2c3-d3696a3d0f2f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:57.853708 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-811f5391-6b31-4746-b2c3-d3696a3d0f2f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:57.854275 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-811f5391-6b31-4746-b2c3-d3696a3d0f2f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:57.855220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-811f5391-6b31-4746-b2c3-d3696a3d0f2f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:57.855883 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-811f5391-6b31-4746-b2c3-d3696a3d0f2f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:57.856519 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-811f5391-6b31-4746-b2c3-d3696a3d0f2f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:58.015252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ff559f6e-9778-43d5-8e04-1b6f96143f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:58.016365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ff559f6e-9778-43d5-8e04-1b6f96143f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:58.025837 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ff559f6e-9778-43d5-8e04-1b6f96143f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:58.028681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ff559f6e-9778-43d5-8e04-1b6f96143f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:58.028681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ff559f6e-9778-43d5-8e04-1b6f96143f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:58.028681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ff559f6e-9778-43d5-8e04-1b6f96143f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:58.029903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ff559f6e-9778-43d5-8e04-1b6f96143f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:58.030385 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ff559f6e-9778-43d5-8e04-1b6f96143f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:58.227633 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-811f5391-6b31-4746-b2c3-d3696a3d0f2f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:58.231774 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ff559f6e-9778-43d5-8e04-1b6f96143f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:58.238700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-811f5391-6b31-4746-b2c3-d3696a3d0f2f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1365 microversion: 2.1 time: 0.397946 Sep 08 13:56:58.240385 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 189/379] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:57 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1365 bytes in 402 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:58.260146 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ff559f6e-9778-43d5-8e04-1b6f96143f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:58.261589 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ff559f6e-9778-43d5-8e04-1b6f96143f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:58.262859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ff559f6e-9778-43d5-8e04-1b6f96143f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1720 microversion: 2.20 time: 0.253313 Sep 08 13:56:58.263419 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 191/380] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:58 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1720 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:56:58.790792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-421577ff-82af-48a0-9c06-d816519cb528 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:58.791698 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-421577ff-82af-48a0-9c06-d816519cb528 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:58.798355 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-421577ff-82af-48a0-9c06-d816519cb528 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:58.799130 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-421577ff-82af-48a0-9c06-d816519cb528 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:58.799746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-421577ff-82af-48a0-9c06-d816519cb528 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:58.800633 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-421577ff-82af-48a0-9c06-d816519cb528 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:58.801249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-421577ff-82af-48a0-9c06-d816519cb528 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:58.801829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-421577ff-82af-48a0-9c06-d816519cb528 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:58.963284 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-421577ff-82af-48a0-9c06-d816519cb528 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:58.971279 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-421577ff-82af-48a0-9c06-d816519cb528 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1319 microversion: 2.1 time: 0.184853 Sep 08 13:56:58.971832 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 190/381] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:58 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1319 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:59.264457 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-06bc3d6d-e4d9-43ca-ab56-779c885ef568 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:59.266112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-06bc3d6d-e4d9-43ca-ab56-779c885ef568 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:59.274715 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06bc3d6d-e4d9-43ca-ab56-779c885ef568 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:59.275557 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06bc3d6d-e4d9-43ca-ab56-779c885ef568 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:59.275995 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06bc3d6d-e4d9-43ca-ab56-779c885ef568 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:59.276867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06bc3d6d-e4d9-43ca-ab56-779c885ef568 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:59.277331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06bc3d6d-e4d9-43ca-ab56-779c885ef568 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:59.277764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06bc3d6d-e4d9-43ca-ab56-779c885ef568 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:59.294901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c5c24c23-17cd-4820-ad19-58d22d927001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:59.297476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c5c24c23-17cd-4820-ad19-58d22d927001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:56:59.304773 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5c24c23-17cd-4820-ad19-58d22d927001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:59.305348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5c24c23-17cd-4820-ad19-58d22d927001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:59.305786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5c24c23-17cd-4820-ad19-58d22d927001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:59.306525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5c24c23-17cd-4820-ad19-58d22d927001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:56:59.307281 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5c24c23-17cd-4820-ad19-58d22d927001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:56:59.307871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5c24c23-17cd-4820-ad19-58d22d927001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:56:59.498256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-06bc3d6d-e4d9-43ca-ab56-779c885ef568 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:59.516392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-06bc3d6d-e4d9-43ca-ab56-779c885ef568 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1365 microversion: 2.1 time: 0.249660 Sep 08 13:56:59.525778 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 192/382] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:59 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1365 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:56:59.617835 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c5c24c23-17cd-4820-ad19-58d22d927001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:59.649440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c5c24c23-17cd-4820-ad19-58d22d927001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:59.653537 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c5c24c23-17cd-4820-ad19-58d22d927001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:56:59.655896 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c5c24c23-17cd-4820-ad19-58d22d927001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1720 microversion: 2.20 time: 0.365135 Sep 08 13:56:59.671841 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 191/383] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:56:59 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1720 bytes in 382 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:56:59.998802 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5b20c3c9-1b48-42a2-8e34-d0559e4e771c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:56:59.999416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5b20c3c9-1b48-42a2-8e34-d0559e4e771c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:00.010350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b20c3c9-1b48-42a2-8e34-d0559e4e771c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:00.011292 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b20c3c9-1b48-42a2-8e34-d0559e4e771c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:00.016643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b20c3c9-1b48-42a2-8e34-d0559e4e771c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:00.016643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b20c3c9-1b48-42a2-8e34-d0559e4e771c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:00.016643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b20c3c9-1b48-42a2-8e34-d0559e4e771c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:00.016643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b20c3c9-1b48-42a2-8e34-d0559e4e771c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:00.129507 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-91988d35-7b73-4081-8e9a-e4d7a8483d56 req-2d954861-5d8f-472b-9f6b-5b019443b8ca service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7ec70561-b436-4d03-bd60-6c5ae232b492", "name": "network-vif-plugged", "status": "completed", "tag": "238b9d36-b260-4554-b58e-6b5beab1736c"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:57:00.149154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-91988d35-7b73-4081-8e9a-e4d7a8483d56 req-2d954861-5d8f-472b-9f6b-5b019443b8ca service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:00.154798 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-91988d35-7b73-4081-8e9a-e4d7a8483d56 req-2d954861-5d8f-472b-9f6b-5b019443b8ca service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:00.154798 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-91988d35-7b73-4081-8e9a-e4d7a8483d56 req-2d954861-5d8f-472b-9f6b-5b019443b8ca service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:00.190508 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-91988d35-7b73-4081-8e9a-e4d7a8483d56 req-2d954861-5d8f-472b-9f6b-5b019443b8ca service nova] Creating event network-vif-plugged:238b9d36-b260-4554-b58e-6b5beab1736c for instance 7ec70561-b436-4d03-bd60-6c5ae232b492 on hv-cinder-85646 Sep 08 13:57:00.207072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-91988d35-7b73-4081-8e9a-e4d7a8483d56 req-2d954861-5d8f-472b-9f6b-5b019443b8ca service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.079096 Sep 08 13:57:00.208181 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 192/384] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:57:00 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:57:00.257411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5b20c3c9-1b48-42a2-8e34-d0559e4e771c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:00.268588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5b20c3c9-1b48-42a2-8e34-d0559e4e771c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1319 microversion: 2.1 time: 0.275568 Sep 08 13:57:00.283913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 193/385] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:56:59 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1319 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:00.538010 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4f1931f0-156a-42be-89f0-38b17d2fbdb0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:00.539489 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4f1931f0-156a-42be-89f0-38b17d2fbdb0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:00.547124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f1931f0-156a-42be-89f0-38b17d2fbdb0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:00.547124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f1931f0-156a-42be-89f0-38b17d2fbdb0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:00.548525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f1931f0-156a-42be-89f0-38b17d2fbdb0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:00.549251 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f1931f0-156a-42be-89f0-38b17d2fbdb0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:00.549709 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f1931f0-156a-42be-89f0-38b17d2fbdb0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:00.550131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f1931f0-156a-42be-89f0-38b17d2fbdb0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:00.695311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-531e6771-3be0-4e10-bcfc-5df60d6071fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:00.696422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-531e6771-3be0-4e10-bcfc-5df60d6071fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:00.722599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-531e6771-3be0-4e10-bcfc-5df60d6071fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:00.728080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-531e6771-3be0-4e10-bcfc-5df60d6071fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:00.728080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-531e6771-3be0-4e10-bcfc-5df60d6071fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:00.728080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-531e6771-3be0-4e10-bcfc-5df60d6071fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:00.728080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-531e6771-3be0-4e10-bcfc-5df60d6071fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:00.728487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-531e6771-3be0-4e10-bcfc-5df60d6071fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:00.731040 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4f1931f0-156a-42be-89f0-38b17d2fbdb0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:00.765906 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4f1931f0-156a-42be-89f0-38b17d2fbdb0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1365 microversion: 2.1 time: 0.229468 Sep 08 13:57:00.782571 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 193/386] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:00 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1365 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:00.910126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-531e6771-3be0-4e10-bcfc-5df60d6071fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:00.919453 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-531e6771-3be0-4e10-bcfc-5df60d6071fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:00.920917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-531e6771-3be0-4e10-bcfc-5df60d6071fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:00.922087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-531e6771-3be0-4e10-bcfc-5df60d6071fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1720 microversion: 2.20 time: 0.231152 Sep 08 13:57:00.922662 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 194/387] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:00 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1720 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:01.297636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fe6a0337-3ce4-4640-bcd5-adce42c58e17 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:01.302627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fe6a0337-3ce4-4640-bcd5-adce42c58e17 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:01.311969 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fe6a0337-3ce4-4640-bcd5-adce42c58e17 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:01.312686 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fe6a0337-3ce4-4640-bcd5-adce42c58e17 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:01.313361 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fe6a0337-3ce4-4640-bcd5-adce42c58e17 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:01.314565 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fe6a0337-3ce4-4640-bcd5-adce42c58e17 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:01.315068 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fe6a0337-3ce4-4640-bcd5-adce42c58e17 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:01.315895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fe6a0337-3ce4-4640-bcd5-adce42c58e17 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:01.494534 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-fe6a0337-3ce4-4640-bcd5-adce42c58e17 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:01.503150 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fe6a0337-3ce4-4640-bcd5-adce42c58e17 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1319 microversion: 2.1 time: 0.212830 Sep 08 13:57:01.504122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 194/388] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:01 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1319 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:01.801540 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-02140031-71b4-4157-8740-324055353834 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:01.802406 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-02140031-71b4-4157-8740-324055353834 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:01.809661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02140031-71b4-4157-8740-324055353834 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:01.810491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02140031-71b4-4157-8740-324055353834 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:01.812202 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02140031-71b4-4157-8740-324055353834 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:01.812922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02140031-71b4-4157-8740-324055353834 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:01.813375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02140031-71b4-4157-8740-324055353834 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:01.814163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02140031-71b4-4157-8740-324055353834 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:01.949602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7b088c9f-37a4-4b21-9984-522c1e7cc888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:01.950821 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7b088c9f-37a4-4b21-9984-522c1e7cc888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:01.960629 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b088c9f-37a4-4b21-9984-522c1e7cc888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:01.961749 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b088c9f-37a4-4b21-9984-522c1e7cc888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:01.962601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b088c9f-37a4-4b21-9984-522c1e7cc888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:01.963861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b088c9f-37a4-4b21-9984-522c1e7cc888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:01.964529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b088c9f-37a4-4b21-9984-522c1e7cc888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:01.965578 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b088c9f-37a4-4b21-9984-522c1e7cc888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:02.141379 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-02140031-71b4-4157-8740-324055353834 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:02.156206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-02140031-71b4-4157-8740-324055353834 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1555 microversion: 2.1 time: 0.359108 Sep 08 13:57:02.158023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 195/389] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:01 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1555 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:02.181217 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-84938cf4-79fe-4647-8e3a-d14640acc192 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:02.182114 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-84938cf4-79fe-4647-8e3a-d14640acc192 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:02.192941 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84938cf4-79fe-4647-8e3a-d14640acc192 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:02.193680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84938cf4-79fe-4647-8e3a-d14640acc192 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:02.194413 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84938cf4-79fe-4647-8e3a-d14640acc192 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:02.204099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84938cf4-79fe-4647-8e3a-d14640acc192 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:02.204099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84938cf4-79fe-4647-8e3a-d14640acc192 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:02.204099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84938cf4-79fe-4647-8e3a-d14640acc192 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:02.291074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-84938cf4-79fe-4647-8e3a-d14640acc192 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.113903 Sep 08 13:57:02.294978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 196/390] 10.222.0.25 () {58 vars in 1275 bytes} [Thu Sep 8 13:57:02 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005/os-interface => generated 285 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:57:02.300062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-4738da58-98f3-4446-a47d-d7a68adc0f77 req-d3c885e6-5aec-4da7-a865-a05a0e756aec service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c14f6817-e3ab-4a36-ba23-590498bf1b25", "name": "network-vif-unplugged", "status": "completed", "tag": "4894f431-b6f6-46b3-95db-e9eb88227e33"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:57:02.310364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-4738da58-98f3-4446-a47d-d7a68adc0f77 req-d3c885e6-5aec-4da7-a865-a05a0e756aec service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:02.320007 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-4738da58-98f3-4446-a47d-d7a68adc0f77 req-d3c885e6-5aec-4da7-a865-a05a0e756aec service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:02.321911 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-4738da58-98f3-4446-a47d-d7a68adc0f77 req-d3c885e6-5aec-4da7-a865-a05a0e756aec service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:02.340732 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-4738da58-98f3-4446-a47d-d7a68adc0f77 req-d3c885e6-5aec-4da7-a865-a05a0e756aec service nova] Creating event network-vif-unplugged:4894f431-b6f6-46b3-95db-e9eb88227e33 for instance c14f6817-e3ab-4a36-ba23-590498bf1b25 on hv-cinder-85646 Sep 08 13:57:02.346560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-4738da58-98f3-4446-a47d-d7a68adc0f77 req-d3c885e6-5aec-4da7-a865-a05a0e756aec service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.049967 Sep 08 13:57:02.347690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 197/391] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:57:02 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:57:02.372309 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7b088c9f-37a4-4b21-9984-522c1e7cc888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:02.373539 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7b088c9f-37a4-4b21-9984-522c1e7cc888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:02.375697 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7b088c9f-37a4-4b21-9984-522c1e7cc888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:02.377855 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7b088c9f-37a4-4b21-9984-522c1e7cc888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1720 microversion: 2.20 time: 0.434389 Sep 08 13:57:02.391261 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 195/392] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:01 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1720 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:02.544421 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f74a3acc-4529-43fd-8aad-af3d37903cf6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:02.546407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f74a3acc-4529-43fd-8aad-af3d37903cf6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:02.579794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f74a3acc-4529-43fd-8aad-af3d37903cf6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:02.588146 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f74a3acc-4529-43fd-8aad-af3d37903cf6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:02.588667 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f74a3acc-4529-43fd-8aad-af3d37903cf6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:02.589438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f74a3acc-4529-43fd-8aad-af3d37903cf6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:02.589954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f74a3acc-4529-43fd-8aad-af3d37903cf6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:02.609144 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f74a3acc-4529-43fd-8aad-af3d37903cf6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:02.967096 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f74a3acc-4529-43fd-8aad-af3d37903cf6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:02.973486 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f74a3acc-4529-43fd-8aad-af3d37903cf6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1319 microversion: 2.1 time: 0.442457 Sep 08 13:57:02.974068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 198/393] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:02 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1319 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:03.417188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2ef88733-975d-453f-97d7-7391bd3fab2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:03.418049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2ef88733-975d-453f-97d7-7391bd3fab2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:03.433046 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2ef88733-975d-453f-97d7-7391bd3fab2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:03.434210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2ef88733-975d-453f-97d7-7391bd3fab2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:03.434842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2ef88733-975d-453f-97d7-7391bd3fab2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:03.435869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2ef88733-975d-453f-97d7-7391bd3fab2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:03.456614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2ef88733-975d-453f-97d7-7391bd3fab2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:03.457182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2ef88733-975d-453f-97d7-7391bd3fab2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.020s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:03.919898 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-2ef88733-975d-453f-97d7-7391bd3fab2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:03.929961 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-2ef88733-975d-453f-97d7-7391bd3fab2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:03.932169 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-2ef88733-975d-453f-97d7-7391bd3fab2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:03.935360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2ef88733-975d-453f-97d7-7391bd3fab2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1723 microversion: 2.20 time: 0.533245 Sep 08 13:57:03.946865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 196/394] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:03 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1723 bytes in 541 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:03.952861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-1803d1d1-9b72-4a1d-b245-e98208967041 req-e31816a3-d932-46b0-8e33-a6eaa98da178 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "82890cea-b89a-4020-b9aa-8a7d94ac6005", "tag": "c152a3b2-3cdc-424f-9020-7dba12d0cb18"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:57:03.967318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-1803d1d1-9b72-4a1d-b245-e98208967041 req-e31816a3-d932-46b0-8e33-a6eaa98da178 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:03.968517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-1803d1d1-9b72-4a1d-b245-e98208967041 req-e31816a3-d932-46b0-8e33-a6eaa98da178 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:03.969115 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-1803d1d1-9b72-4a1d-b245-e98208967041 req-e31816a3-d932-46b0-8e33-a6eaa98da178 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:03.981890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-37e29f17-e87b-4c7b-a323-504a3c5390c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "781bf9e8-460d-434e-8629-f5fd94f42731"}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:57:03.984811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-37e29f17-e87b-4c7b-a323-504a3c5390c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:03.992879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-37e29f17-e87b-4c7b-a323-504a3c5390c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:03.993988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-37e29f17-e87b-4c7b-a323-504a3c5390c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:03.994739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-37e29f17-e87b-4c7b-a323-504a3c5390c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:03.996180 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-37e29f17-e87b-4c7b-a323-504a3c5390c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:03.997020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-37e29f17-e87b-4c7b-a323-504a3c5390c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:03.997278 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-1803d1d1-9b72-4a1d-b245-e98208967041 req-e31816a3-d932-46b0-8e33-a6eaa98da178 service nova] Creating event network-changed:c152a3b2-3cdc-424f-9020-7dba12d0cb18 for instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 on hv-cinder-85646 Sep 08 13:57:03.997882 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-37e29f17-e87b-4c7b-a323-504a3c5390c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:04.005399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-1803d1d1-9b72-4a1d-b245-e98208967041 req-e31816a3-d932-46b0-8e33-a6eaa98da178 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.056422 Sep 08 13:57:04.006027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 199/395] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:57:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:57:04.014685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6465ac20-f01c-472c-b99c-d0cc5083d967 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:04.015783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6465ac20-f01c-472c-b99c-d0cc5083d967 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:04.030434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6465ac20-f01c-472c-b99c-d0cc5083d967 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:04.038180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6465ac20-f01c-472c-b99c-d0cc5083d967 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:04.038748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6465ac20-f01c-472c-b99c-d0cc5083d967 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:04.039485 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6465ac20-f01c-472c-b99c-d0cc5083d967 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:04.044705 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6465ac20-f01c-472c-b99c-d0cc5083d967 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:04.045703 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6465ac20-f01c-472c-b99c-d0cc5083d967 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:04.334467 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6465ac20-f01c-472c-b99c-d0cc5083d967 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:04.357048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6465ac20-f01c-472c-b99c-d0cc5083d967 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1319 microversion: 2.1 time: 0.336525 Sep 08 13:57:04.357048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 200/396] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:04 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1319 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:05.379809 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d97046f6-5211-4574-bcca-edf0c5b839a2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:05.381310 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d97046f6-5211-4574-bcca-edf0c5b839a2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:05.391344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d97046f6-5211-4574-bcca-edf0c5b839a2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:05.392260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d97046f6-5211-4574-bcca-edf0c5b839a2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:05.393039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d97046f6-5211-4574-bcca-edf0c5b839a2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:05.395152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d97046f6-5211-4574-bcca-edf0c5b839a2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:05.395725 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d97046f6-5211-4574-bcca-edf0c5b839a2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:05.396438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d97046f6-5211-4574-bcca-edf0c5b839a2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:05.403248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:05.618916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d97046f6-5211-4574-bcca-edf0c5b839a2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:05.626986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d97046f6-5211-4574-bcca-edf0c5b839a2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1507 microversion: 2.1 time: 0.251908 Sep 08 13:57:05.633897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-37e29f17-e87b-4c7b-a323-504a3c5390c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "POST /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 1.657353 Sep 08 13:57:05.643516 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 197/397] 10.222.0.25 () {62 vars in 1364 bytes} [Thu Sep 8 13:57:03 2022] POST /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 188 bytes in 1660 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:57:05.644774 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 201/398] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:05 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1507 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:06.205578 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-be053ad1-4792-4946-bc75-a59adc10fbac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:57:06.207781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-be053ad1-4792-4946-bc75-a59adc10fbac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:06.231427 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be053ad1-4792-4946-bc75-a59adc10fbac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:06.232600 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be053ad1-4792-4946-bc75-a59adc10fbac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:06.232894 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be053ad1-4792-4946-bc75-a59adc10fbac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:06.233833 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be053ad1-4792-4946-bc75-a59adc10fbac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:06.237119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be053ad1-4792-4946-bc75-a59adc10fbac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:06.237735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be053ad1-4792-4946-bc75-a59adc10fbac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:06.333568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-be053ad1-4792-4946-bc75-a59adc10fbac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=127602) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 08 13:57:06.492874 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=92,nova_cell1:INSERT=3,nova_cell1:UPDATE=5 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:06.776003 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-be053ad1-4792-4946-bc75-a59adc10fbac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "POST /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/action" status: 202 len: 0 microversion: 2.20 time: 0.587770 Sep 08 13:57:06.780565 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 198/399] 10.222.0.25 () {62 vars in 1319 bytes} [Thu Sep 8 13:57:06 2022] POST /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/action => generated 0 bytes in 593 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 08 13:57:06.788857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8107c6a9-70b6-4006-a918-590268f17e15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:06.789481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8107c6a9-70b6-4006-a918-590268f17e15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:06.803539 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8107c6a9-70b6-4006-a918-590268f17e15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:06.805493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8107c6a9-70b6-4006-a918-590268f17e15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:06.807838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8107c6a9-70b6-4006-a918-590268f17e15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:06.808891 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8107c6a9-70b6-4006-a918-590268f17e15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:06.811299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8107c6a9-70b6-4006-a918-590268f17e15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:06.811826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8107c6a9-70b6-4006-a918-590268f17e15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:07.047733 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8107c6a9-70b6-4006-a918-590268f17e15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:07.060032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8107c6a9-70b6-4006-a918-590268f17e15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:07.061864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8107c6a9-70b6-4006-a918-590268f17e15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:07.063261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8107c6a9-70b6-4006-a918-590268f17e15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1749 microversion: 2.20 time: 0.278578 Sep 08 13:57:07.063990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 202/400] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:06 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1749 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:08.089646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-61e8e96f-8a77-4226-88bd-00eaa0f8bd25 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:08.090540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-61e8e96f-8a77-4226-88bd-00eaa0f8bd25 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:08.101336 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-61e8e96f-8a77-4226-88bd-00eaa0f8bd25 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:08.103518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-61e8e96f-8a77-4226-88bd-00eaa0f8bd25 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:08.104084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-61e8e96f-8a77-4226-88bd-00eaa0f8bd25 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:08.105472 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-61e8e96f-8a77-4226-88bd-00eaa0f8bd25 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:08.106017 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-61e8e96f-8a77-4226-88bd-00eaa0f8bd25 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:08.106673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-61e8e96f-8a77-4226-88bd-00eaa0f8bd25 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:08.328767 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-61e8e96f-8a77-4226-88bd-00eaa0f8bd25 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:08.338055 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-61e8e96f-8a77-4226-88bd-00eaa0f8bd25 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:08.339678 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-61e8e96f-8a77-4226-88bd-00eaa0f8bd25 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:08.340977 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-61e8e96f-8a77-4226-88bd-00eaa0f8bd25 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1749 microversion: 2.20 time: 0.255500 Sep 08 13:57:08.341628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 199/401] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:08 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1749 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:09.369270 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-20fe537c-fea0-49ad-afd6-0ff006ae3333 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:09.370053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-20fe537c-fea0-49ad-afd6-0ff006ae3333 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:09.379504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-20fe537c-fea0-49ad-afd6-0ff006ae3333 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:09.380017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-20fe537c-fea0-49ad-afd6-0ff006ae3333 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:09.380459 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-20fe537c-fea0-49ad-afd6-0ff006ae3333 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:09.381363 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-20fe537c-fea0-49ad-afd6-0ff006ae3333 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:09.382013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-20fe537c-fea0-49ad-afd6-0ff006ae3333 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:09.382652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-20fe537c-fea0-49ad-afd6-0ff006ae3333 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:09.572770 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-20fe537c-fea0-49ad-afd6-0ff006ae3333 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:09.580143 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-20fe537c-fea0-49ad-afd6-0ff006ae3333 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:09.581846 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-20fe537c-fea0-49ad-afd6-0ff006ae3333 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:09.583264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-20fe537c-fea0-49ad-afd6-0ff006ae3333 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1749 microversion: 2.20 time: 0.220901 Sep 08 13:57:09.583996 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 203/402] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:09 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1749 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:10.611136 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-83f38ec7-8141-459e-a2f4-9930f34e92aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:10.611934 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-83f38ec7-8141-459e-a2f4-9930f34e92aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:10.622417 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-83f38ec7-8141-459e-a2f4-9930f34e92aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:10.622417 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-83f38ec7-8141-459e-a2f4-9930f34e92aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:10.622417 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-83f38ec7-8141-459e-a2f4-9930f34e92aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:10.622417 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-83f38ec7-8141-459e-a2f4-9930f34e92aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:10.622735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-83f38ec7-8141-459e-a2f4-9930f34e92aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:10.622735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-83f38ec7-8141-459e-a2f4-9930f34e92aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:10.789478 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-09e1bcfe-c278-4ee2-9089-ad4c5a71e910 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-814467563", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-720914496", "security_groups": [{"name": "tempest-secgroup-smoke-640822901"}], "block_device_mapping_v2": [{"uuid": "954375a6-1654-46da-915f-ebf0a63048cf", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "ceecd0c0-50df-4c0f-8d9c-07ccfcb1ae0a"}]}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:57:10.794049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-83f38ec7-8141-459e-a2f4-9930f34e92aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:10.807674 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-83f38ec7-8141-459e-a2f4-9930f34e92aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:10.809199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-83f38ec7-8141-459e-a2f4-9930f34e92aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:10.810519 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-83f38ec7-8141-459e-a2f4-9930f34e92aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1749 microversion: 2.20 time: 0.203285 Sep 08 13:57:10.811261 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 200/403] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:10 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1749 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:10.917161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.network.neutron [None req-09e1bcfe-c278-4ee2-9089-ad4c5a71e910 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] validate_networks() for [('ceecd0c0-50df-4c0f-8d9c-07ccfcb1ae0a', None, None, None, None, None)] {{(pid=127601) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 13:57:11.318607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-09e1bcfe-c278-4ee2-9089-ad4c5a71e910 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Flavor limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 13:57:11.319381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-09e1bcfe-c278-4ee2-9089-ad4c5a71e910 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Image limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 13:57:11.319937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-09e1bcfe-c278-4ee2-9089-ad4c5a71e910 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Flavor pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 13:57:11.320414 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-09e1bcfe-c278-4ee2-9089-ad4c5a71e910 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Image pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 13:57:11.320895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-09e1bcfe-c278-4ee2-9089-ad4c5a71e910 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 13:57:11.338448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-09e1bcfe-c278-4ee2-9089-ad4c5a71e910 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:11.338448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-09e1bcfe-c278-4ee2-9089-ad4c5a71e910 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:11.338448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-09e1bcfe-c278-4ee2-9089-ad4c5a71e910 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:11.338448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-09e1bcfe-c278-4ee2-9089-ad4c5a71e910 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:11.338855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-09e1bcfe-c278-4ee2-9089-ad4c5a71e910 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:11.338855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-09e1bcfe-c278-4ee2-9089-ad4c5a71e910 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:11.352153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-09e1bcfe-c278-4ee2-9089-ad4c5a71e910 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Getting quotas for project 45579d93010d4ed0b80cad3a8b85ea50. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:57:11.356391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-09e1bcfe-c278-4ee2-9089-ad4c5a71e910 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Getting quotas for user 772205d93ce047e9ad4ddefadec300f4 and project 45579d93010d4ed0b80cad3a8b85ea50. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:57:11.423633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-09e1bcfe-c278-4ee2-9089-ad4c5a71e910 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Going to run 1 instances... {{(pid=127601) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 13:57:11.425766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-09e1bcfe-c278-4ee2-9089-ad4c5a71e910 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='954375a6-1654-46da-915f-ebf0a63048cf',volume_size=None,volume_type=None)] {{(pid=127601) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 13:57:11.838166 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e27c467e-5cde-47e8-8e32-e0b6fe673599 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:11.839265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e27c467e-5cde-47e8-8e32-e0b6fe673599 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:11.846616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e27c467e-5cde-47e8-8e32-e0b6fe673599 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:11.847215 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e27c467e-5cde-47e8-8e32-e0b6fe673599 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:11.847905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e27c467e-5cde-47e8-8e32-e0b6fe673599 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:11.848797 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e27c467e-5cde-47e8-8e32-e0b6fe673599 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:11.849325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e27c467e-5cde-47e8-8e32-e0b6fe673599 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:11.850022 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e27c467e-5cde-47e8-8e32-e0b6fe673599 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:12.021945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-09e1bcfe-c278-4ee2-9089-ad4c5a71e910 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 1.237831 Sep 08 13:57:12.023469 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 204/404] 10.222.0.25 () {60 vars in 1147 bytes} [Thu Sep 8 13:57:10 2022] POST /compute/v2.1/servers => generated 405 bytes in 1240 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 08 13:57:12.030206 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e27c467e-5cde-47e8-8e32-e0b6fe673599 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:12.034956 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-faa878ea-d436-40d7-937c-0a7522ebf685 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:12.035746 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-faa878ea-d436-40d7-937c-0a7522ebf685 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:12.049525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e27c467e-5cde-47e8-8e32-e0b6fe673599 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:12.051284 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e27c467e-5cde-47e8-8e32-e0b6fe673599 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:12.056088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e27c467e-5cde-47e8-8e32-e0b6fe673599 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1749 microversion: 2.20 time: 0.222680 Sep 08 13:57:12.056662 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 201/405] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:11 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1749 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:12.107320 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-faa878ea-d436-40d7-937c-0a7522ebf685 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:12.115320 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-faa878ea-d436-40d7-937c-0a7522ebf685 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1093 microversion: 2.1 time: 0.084292 Sep 08 13:57:12.115974 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 205/406] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:12 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1093 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:13.099306 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2583f8ca-9dd3-4d2b-bc07-cee17effac3c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:13.100136 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2583f8ca-9dd3-4d2b-bc07-cee17effac3c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:13.103415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=66,nova_api:UPDATE=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:13.106949 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2583f8ca-9dd3-4d2b-bc07-cee17effac3c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:13.107432 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2583f8ca-9dd3-4d2b-bc07-cee17effac3c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:13.107877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2583f8ca-9dd3-4d2b-bc07-cee17effac3c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:13.108568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2583f8ca-9dd3-4d2b-bc07-cee17effac3c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:13.109024 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2583f8ca-9dd3-4d2b-bc07-cee17effac3c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:13.109451 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2583f8ca-9dd3-4d2b-bc07-cee17effac3c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:13.141645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-deaa84e0-25cb-489f-a6b0-c57536a0a83a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:13.145890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-deaa84e0-25cb-489f-a6b0-c57536a0a83a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:13.203632 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-deaa84e0-25cb-489f-a6b0-c57536a0a83a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:13.221732 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-deaa84e0-25cb-489f-a6b0-c57536a0a83a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1093 microversion: 2.1 time: 0.086252 Sep 08 13:57:13.223026 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 206/407] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:13 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1093 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:13.287665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-2583f8ca-9dd3-4d2b-bc07-cee17effac3c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:13.298190 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-2583f8ca-9dd3-4d2b-bc07-cee17effac3c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:13.299794 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-2583f8ca-9dd3-4d2b-bc07-cee17effac3c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:13.301070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2583f8ca-9dd3-4d2b-bc07-cee17effac3c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1749 microversion: 2.20 time: 0.213833 Sep 08 13:57:13.301643 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 202/408] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:13 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1749 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:14.245056 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cf5dba33-7250-4ee7-863b-3ea3974301e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:14.246432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cf5dba33-7250-4ee7-863b-3ea3974301e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:14.253818 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cf5dba33-7250-4ee7-863b-3ea3974301e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:14.254359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cf5dba33-7250-4ee7-863b-3ea3974301e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:14.254969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cf5dba33-7250-4ee7-863b-3ea3974301e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:14.255979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cf5dba33-7250-4ee7-863b-3ea3974301e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:14.256609 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cf5dba33-7250-4ee7-863b-3ea3974301e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:14.257161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cf5dba33-7250-4ee7-863b-3ea3974301e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:14.328781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cf5dba33-7250-4ee7-863b-3ea3974301e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:14.329644 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-92e682a6-d079-46c5-b613-9a9a1aefaa8a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:14.330871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-92e682a6-d079-46c5-b613-9a9a1aefaa8a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:14.338330 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-92e682a6-d079-46c5-b613-9a9a1aefaa8a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:14.339297 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cf5dba33-7250-4ee7-863b-3ea3974301e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1191 microversion: 2.1 time: 0.098179 Sep 08 13:57:14.339434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-92e682a6-d079-46c5-b613-9a9a1aefaa8a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:14.340081 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-92e682a6-d079-46c5-b613-9a9a1aefaa8a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:14.340203 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 207/409] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:14 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1191 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:14.341162 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-92e682a6-d079-46c5-b613-9a9a1aefaa8a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:14.341623 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-92e682a6-d079-46c5-b613-9a9a1aefaa8a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:14.342050 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-92e682a6-d079-46c5-b613-9a9a1aefaa8a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:14.518039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-92e682a6-d079-46c5-b613-9a9a1aefaa8a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:14.527976 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-92e682a6-d079-46c5-b613-9a9a1aefaa8a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:14.529455 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-92e682a6-d079-46c5-b613-9a9a1aefaa8a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:14.530802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-92e682a6-d079-46c5-b613-9a9a1aefaa8a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1749 microversion: 2.20 time: 0.206083 Sep 08 13:57:14.534077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 203/410] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:14 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1749 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:15.365380 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7f61ea95-baa7-45ce-a110-4ae1943a2f14 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:15.365999 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7f61ea95-baa7-45ce-a110-4ae1943a2f14 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:15.373131 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f61ea95-baa7-45ce-a110-4ae1943a2f14 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:15.373623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f61ea95-baa7-45ce-a110-4ae1943a2f14 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:15.374065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f61ea95-baa7-45ce-a110-4ae1943a2f14 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:15.374789 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f61ea95-baa7-45ce-a110-4ae1943a2f14 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:15.375247 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f61ea95-baa7-45ce-a110-4ae1943a2f14 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:15.375676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f61ea95-baa7-45ce-a110-4ae1943a2f14 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:15.439852 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7f61ea95-baa7-45ce-a110-4ae1943a2f14 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:15.450205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7f61ea95-baa7-45ce-a110-4ae1943a2f14 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1199 microversion: 2.1 time: 0.089257 Sep 08 13:57:15.467510 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 208/411] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:15 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1199 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:15.569970 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3918ddf7-6003-4782-9d5b-0aa732a8b2b2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:15.570734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3918ddf7-6003-4782-9d5b-0aa732a8b2b2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:15.577540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3918ddf7-6003-4782-9d5b-0aa732a8b2b2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:15.578033 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3918ddf7-6003-4782-9d5b-0aa732a8b2b2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:15.578569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3918ddf7-6003-4782-9d5b-0aa732a8b2b2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:15.579311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3918ddf7-6003-4782-9d5b-0aa732a8b2b2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:15.579756 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3918ddf7-6003-4782-9d5b-0aa732a8b2b2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:15.580362 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3918ddf7-6003-4782-9d5b-0aa732a8b2b2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:15.852533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3918ddf7-6003-4782-9d5b-0aa732a8b2b2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:15.862687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3918ddf7-6003-4782-9d5b-0aa732a8b2b2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:15.864536 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3918ddf7-6003-4782-9d5b-0aa732a8b2b2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:15.867628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3918ddf7-6003-4782-9d5b-0aa732a8b2b2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1749 microversion: 2.20 time: 0.303390 Sep 08 13:57:15.868387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 204/412] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:15 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1749 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:16.475026 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7031c0d9-3dc4-4cd7-a6e1-3da55bc23d4b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:16.475490 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7031c0d9-3dc4-4cd7-a6e1-3da55bc23d4b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:16.482112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7031c0d9-3dc4-4cd7-a6e1-3da55bc23d4b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:16.482732 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7031c0d9-3dc4-4cd7-a6e1-3da55bc23d4b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:16.483168 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7031c0d9-3dc4-4cd7-a6e1-3da55bc23d4b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:16.483864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7031c0d9-3dc4-4cd7-a6e1-3da55bc23d4b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:16.484320 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7031c0d9-3dc4-4cd7-a6e1-3da55bc23d4b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:16.484749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7031c0d9-3dc4-4cd7-a6e1-3da55bc23d4b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:16.637205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7031c0d9-3dc4-4cd7-a6e1-3da55bc23d4b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:16.646583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7031c0d9-3dc4-4cd7-a6e1-3da55bc23d4b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1276 microversion: 2.1 time: 0.175881 Sep 08 13:57:16.647179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 209/413] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:16 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1276 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:16.894896 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-01967f4a-845a-483b-8e8e-ca65ae00d3fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:16.895611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-01967f4a-845a-483b-8e8e-ca65ae00d3fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:16.903073 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-01967f4a-845a-483b-8e8e-ca65ae00d3fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:16.912445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-01967f4a-845a-483b-8e8e-ca65ae00d3fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:16.912445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-01967f4a-845a-483b-8e8e-ca65ae00d3fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:16.912445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-01967f4a-845a-483b-8e8e-ca65ae00d3fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:16.912445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-01967f4a-845a-483b-8e8e-ca65ae00d3fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:16.912850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-01967f4a-845a-483b-8e8e-ca65ae00d3fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:17.056783 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-01967f4a-845a-483b-8e8e-ca65ae00d3fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:17.066595 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-01967f4a-845a-483b-8e8e-ca65ae00d3fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:17.068294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-01967f4a-845a-483b-8e8e-ca65ae00d3fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:17.069673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-01967f4a-845a-483b-8e8e-ca65ae00d3fd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1749 microversion: 2.20 time: 0.178685 Sep 08 13:57:17.070940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 205/414] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:16 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1749 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:17.493397 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-ad48666c-aecf-4261-940f-bc697c4e2559 req-43c1702e-2eee-4c2d-a8ad-bb25f002e82b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6c3f043c-0662-4b63-bbd5-3a0f68c89b94", "tag": "0476c717-e075-4f94-b8e8-3fa5f88138b1"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:57:17.506997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-ad48666c-aecf-4261-940f-bc697c4e2559 req-43c1702e-2eee-4c2d-a8ad-bb25f002e82b service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:17.507636 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-ad48666c-aecf-4261-940f-bc697c4e2559 req-43c1702e-2eee-4c2d-a8ad-bb25f002e82b service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:17.508253 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-ad48666c-aecf-4261-940f-bc697c4e2559 req-43c1702e-2eee-4c2d-a8ad-bb25f002e82b service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:17.521274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-ad48666c-aecf-4261-940f-bc697c4e2559 req-43c1702e-2eee-4c2d-a8ad-bb25f002e82b service nova] Creating event network-changed:0476c717-e075-4f94-b8e8-3fa5f88138b1 for instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 on hv-cinder-85646 Sep 08 13:57:17.526756 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-ad48666c-aecf-4261-940f-bc697c4e2559 req-43c1702e-2eee-4c2d-a8ad-bb25f002e82b service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038803 Sep 08 13:57:17.527513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 210/415] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:57:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:57:17.670736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f9eaf31f-8c1e-41cc-bddb-e1d07ecb8dcf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:17.671388 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f9eaf31f-8c1e-41cc-bddb-e1d07ecb8dcf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:17.677993 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f9eaf31f-8c1e-41cc-bddb-e1d07ecb8dcf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:17.678543 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f9eaf31f-8c1e-41cc-bddb-e1d07ecb8dcf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:17.679658 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f9eaf31f-8c1e-41cc-bddb-e1d07ecb8dcf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:17.680374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f9eaf31f-8c1e-41cc-bddb-e1d07ecb8dcf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:17.681085 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f9eaf31f-8c1e-41cc-bddb-e1d07ecb8dcf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:17.681571 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f9eaf31f-8c1e-41cc-bddb-e1d07ecb8dcf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:17.920183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f9eaf31f-8c1e-41cc-bddb-e1d07ecb8dcf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:17.929404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f9eaf31f-8c1e-41cc-bddb-e1d07ecb8dcf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1276 microversion: 2.1 time: 0.262210 Sep 08 13:57:17.930621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 206/416] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:17 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1276 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:18.102466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1d8756de-c702-4cce-8243-532049bcca4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:18.103863 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1d8756de-c702-4cce-8243-532049bcca4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:18.110898 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d8756de-c702-4cce-8243-532049bcca4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:18.111506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d8756de-c702-4cce-8243-532049bcca4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:18.112006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d8756de-c702-4cce-8243-532049bcca4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:18.112823 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d8756de-c702-4cce-8243-532049bcca4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:18.113349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d8756de-c702-4cce-8243-532049bcca4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:18.113956 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d8756de-c702-4cce-8243-532049bcca4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:18.299233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1d8756de-c702-4cce-8243-532049bcca4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:18.308552 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1d8756de-c702-4cce-8243-532049bcca4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:18.309982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1d8756de-c702-4cce-8243-532049bcca4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:18.311178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1d8756de-c702-4cce-8243-532049bcca4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1749 microversion: 2.20 time: 0.214183 Sep 08 13:57:18.311763 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 211/417] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:18 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1749 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:18.953052 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5678e7cb-643f-4377-9f17-c7e3d991b250 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:18.954439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5678e7cb-643f-4377-9f17-c7e3d991b250 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:18.961399 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5678e7cb-643f-4377-9f17-c7e3d991b250 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:18.965264 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5678e7cb-643f-4377-9f17-c7e3d991b250 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:18.966206 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5678e7cb-643f-4377-9f17-c7e3d991b250 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:18.967253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5678e7cb-643f-4377-9f17-c7e3d991b250 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:18.968028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5678e7cb-643f-4377-9f17-c7e3d991b250 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:18.968607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5678e7cb-643f-4377-9f17-c7e3d991b250 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:19.166867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5678e7cb-643f-4377-9f17-c7e3d991b250 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:19.177847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5678e7cb-643f-4377-9f17-c7e3d991b250 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.226395 Sep 08 13:57:19.179892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 207/418] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:18 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:19.338877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-55fd473a-f620-40da-a29a-d5188e1a93cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:19.339583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-55fd473a-f620-40da-a29a-d5188e1a93cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:19.348975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-55fd473a-f620-40da-a29a-d5188e1a93cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:19.349688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-55fd473a-f620-40da-a29a-d5188e1a93cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:19.350546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-55fd473a-f620-40da-a29a-d5188e1a93cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:19.351978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-55fd473a-f620-40da-a29a-d5188e1a93cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:19.352609 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-55fd473a-f620-40da-a29a-d5188e1a93cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:19.353227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-55fd473a-f620-40da-a29a-d5188e1a93cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:19.511380 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-55fd473a-f620-40da-a29a-d5188e1a93cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:19.516685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-55fd473a-f620-40da-a29a-d5188e1a93cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:19.518214 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-55fd473a-f620-40da-a29a-d5188e1a93cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:19.519582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-55fd473a-f620-40da-a29a-d5188e1a93cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1749 microversion: 2.20 time: 0.186200 Sep 08 13:57:19.530621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 212/419] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:19 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1749 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:20.213401 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a5254320-cc5e-421b-a202-eb5601d571fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:20.215503 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a5254320-cc5e-421b-a202-eb5601d571fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:20.225438 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5254320-cc5e-421b-a202-eb5601d571fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:20.226863 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5254320-cc5e-421b-a202-eb5601d571fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:20.227334 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5254320-cc5e-421b-a202-eb5601d571fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:20.228273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5254320-cc5e-421b-a202-eb5601d571fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:20.229037 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5254320-cc5e-421b-a202-eb5601d571fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:20.229605 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5254320-cc5e-421b-a202-eb5601d571fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:20.468735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a5254320-cc5e-421b-a202-eb5601d571fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:20.487445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a5254320-cc5e-421b-a202-eb5601d571fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.284781 Sep 08 13:57:20.502844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 208/420] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:20 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:20.551039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-57d85282-5c65-4a58-9171-e4cae6951ecd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:20.551739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-57d85282-5c65-4a58-9171-e4cae6951ecd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:20.558534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57d85282-5c65-4a58-9171-e4cae6951ecd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:20.559302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57d85282-5c65-4a58-9171-e4cae6951ecd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:20.559906 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57d85282-5c65-4a58-9171-e4cae6951ecd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:20.560792 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57d85282-5c65-4a58-9171-e4cae6951ecd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:20.561400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57d85282-5c65-4a58-9171-e4cae6951ecd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:20.562289 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57d85282-5c65-4a58-9171-e4cae6951ecd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:20.750933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-57d85282-5c65-4a58-9171-e4cae6951ecd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:20.761429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-57d85282-5c65-4a58-9171-e4cae6951ecd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:20.763443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-57d85282-5c65-4a58-9171-e4cae6951ecd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:20.764748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-57d85282-5c65-4a58-9171-e4cae6951ecd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1751 microversion: 2.20 time: 0.217566 Sep 08 13:57:20.778800 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 213/421] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:20 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1751 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:21.512056 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-89fedbaf-87bd-4143-a9cd-eba93398fafb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:21.513344 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-89fedbaf-87bd-4143-a9cd-eba93398fafb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:21.521561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89fedbaf-87bd-4143-a9cd-eba93398fafb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:21.522053 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89fedbaf-87bd-4143-a9cd-eba93398fafb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:21.522518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89fedbaf-87bd-4143-a9cd-eba93398fafb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:21.523772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89fedbaf-87bd-4143-a9cd-eba93398fafb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:21.524214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89fedbaf-87bd-4143-a9cd-eba93398fafb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:21.524806 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89fedbaf-87bd-4143-a9cd-eba93398fafb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:21.719515 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-89fedbaf-87bd-4143-a9cd-eba93398fafb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:21.719515 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-89fedbaf-87bd-4143-a9cd-eba93398fafb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.195116 Sep 08 13:57:21.719515 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 209/422] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:21 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:21.793658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c9e9f2b9-23d2-4827-aed9-be446b17d113 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:21.794421 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c9e9f2b9-23d2-4827-aed9-be446b17d113 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:21.798798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=2 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:21.803197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c9e9f2b9-23d2-4827-aed9-be446b17d113 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:21.803683 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c9e9f2b9-23d2-4827-aed9-be446b17d113 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:21.804121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c9e9f2b9-23d2-4827-aed9-be446b17d113 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:21.804819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c9e9f2b9-23d2-4827-aed9-be446b17d113 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:21.805272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c9e9f2b9-23d2-4827-aed9-be446b17d113 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:21.805697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c9e9f2b9-23d2-4827-aed9-be446b17d113 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:21.956048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c9e9f2b9-23d2-4827-aed9-be446b17d113 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:21.965815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c9e9f2b9-23d2-4827-aed9-be446b17d113 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:21.967325 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c9e9f2b9-23d2-4827-aed9-be446b17d113 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:21.968541 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c9e9f2b9-23d2-4827-aed9-be446b17d113 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1751 microversion: 2.20 time: 0.178749 Sep 08 13:57:21.969029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 214/423] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:21 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1751 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:22.736013 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e7bed5b4-40b6-4c9f-95cb-6e087ed12935 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:22.738292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e7bed5b4-40b6-4c9f-95cb-6e087ed12935 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:22.749721 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7bed5b4-40b6-4c9f-95cb-6e087ed12935 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:22.750809 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7bed5b4-40b6-4c9f-95cb-6e087ed12935 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:22.751632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7bed5b4-40b6-4c9f-95cb-6e087ed12935 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:22.752685 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7bed5b4-40b6-4c9f-95cb-6e087ed12935 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:22.753359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7bed5b4-40b6-4c9f-95cb-6e087ed12935 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:22.754294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7bed5b4-40b6-4c9f-95cb-6e087ed12935 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:22.945171 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e7bed5b4-40b6-4c9f-95cb-6e087ed12935 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:22.953842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e7bed5b4-40b6-4c9f-95cb-6e087ed12935 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.229762 Sep 08 13:57:22.954453 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 210/424] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:22 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:22.995527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5f576409-b562-4736-801e-34cfeb523d48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:22.996903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5f576409-b562-4736-801e-34cfeb523d48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:23.004236 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f576409-b562-4736-801e-34cfeb523d48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:23.004707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f576409-b562-4736-801e-34cfeb523d48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:23.005151 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f576409-b562-4736-801e-34cfeb523d48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:23.005860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f576409-b562-4736-801e-34cfeb523d48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:23.006368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f576409-b562-4736-801e-34cfeb523d48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:23.006968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f576409-b562-4736-801e-34cfeb523d48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:23.175935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5f576409-b562-4736-801e-34cfeb523d48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:23.187775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5f576409-b562-4736-801e-34cfeb523d48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:23.193102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5f576409-b562-4736-801e-34cfeb523d48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:23.193102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5f576409-b562-4736-801e-34cfeb523d48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1751 microversion: 2.20 time: 0.199705 Sep 08 13:57:23.193102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 215/425] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:22 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1751 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:23.979343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4b951f73-dda8-4df6-9347-aaa4fad1044b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:23.980590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4b951f73-dda8-4df6-9347-aaa4fad1044b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:23.989252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b951f73-dda8-4df6-9347-aaa4fad1044b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:23.989762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b951f73-dda8-4df6-9347-aaa4fad1044b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:23.990422 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b951f73-dda8-4df6-9347-aaa4fad1044b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:23.991483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b951f73-dda8-4df6-9347-aaa4fad1044b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:23.991950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b951f73-dda8-4df6-9347-aaa4fad1044b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:23.992382 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b951f73-dda8-4df6-9347-aaa4fad1044b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:24.192317 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4b951f73-dda8-4df6-9347-aaa4fad1044b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:24.214418 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4b951f73-dda8-4df6-9347-aaa4fad1044b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.229971 Sep 08 13:57:24.214418 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 211/426] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:23 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:24.217284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-edc149a7-96d8-4718-83bb-6f6edcd83fa6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:24.218077 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-edc149a7-96d8-4718-83bb-6f6edcd83fa6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:24.221058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=10 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:24.222404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=87,nova_api:INSERT=3 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:24.226614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-edc149a7-96d8-4718-83bb-6f6edcd83fa6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:24.227412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-edc149a7-96d8-4718-83bb-6f6edcd83fa6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:24.228033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-edc149a7-96d8-4718-83bb-6f6edcd83fa6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:24.228937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-edc149a7-96d8-4718-83bb-6f6edcd83fa6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:24.229621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-edc149a7-96d8-4718-83bb-6f6edcd83fa6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:24.230243 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-edc149a7-96d8-4718-83bb-6f6edcd83fa6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:24.402633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-edc149a7-96d8-4718-83bb-6f6edcd83fa6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:24.412407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-edc149a7-96d8-4718-83bb-6f6edcd83fa6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:24.414929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-edc149a7-96d8-4718-83bb-6f6edcd83fa6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:24.416484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-edc149a7-96d8-4718-83bb-6f6edcd83fa6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.203131 Sep 08 13:57:24.430749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 216/427] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:24 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:25.236266 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3690d02f-e671-4220-9eee-29bda9d8aedb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:25.237092 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3690d02f-e671-4220-9eee-29bda9d8aedb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:25.248799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3690d02f-e671-4220-9eee-29bda9d8aedb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:25.249317 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3690d02f-e671-4220-9eee-29bda9d8aedb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:25.249791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3690d02f-e671-4220-9eee-29bda9d8aedb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:25.250531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3690d02f-e671-4220-9eee-29bda9d8aedb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:25.250987 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3690d02f-e671-4220-9eee-29bda9d8aedb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:25.251412 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3690d02f-e671-4220-9eee-29bda9d8aedb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:25.391241 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3690d02f-e671-4220-9eee-29bda9d8aedb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:25.399044 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3690d02f-e671-4220-9eee-29bda9d8aedb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.171813 Sep 08 13:57:25.399704 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 212/428] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:25 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:25.444581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5aa9e3f8-cd8f-4d24-89a2-dd7ec719e4de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:25.445749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5aa9e3f8-cd8f-4d24-89a2-dd7ec719e4de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:25.452551 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aa9e3f8-cd8f-4d24-89a2-dd7ec719e4de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:25.453041 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aa9e3f8-cd8f-4d24-89a2-dd7ec719e4de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:25.453483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aa9e3f8-cd8f-4d24-89a2-dd7ec719e4de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:25.454259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aa9e3f8-cd8f-4d24-89a2-dd7ec719e4de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:25.454718 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aa9e3f8-cd8f-4d24-89a2-dd7ec719e4de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:25.455544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aa9e3f8-cd8f-4d24-89a2-dd7ec719e4de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:25.643922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5aa9e3f8-cd8f-4d24-89a2-dd7ec719e4de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:25.664423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5aa9e3f8-cd8f-4d24-89a2-dd7ec719e4de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:25.673200 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5aa9e3f8-cd8f-4d24-89a2-dd7ec719e4de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:25.674839 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5aa9e3f8-cd8f-4d24-89a2-dd7ec719e4de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.234454 Sep 08 13:57:25.677333 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 217/429] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:25 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:26.423611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fa5e6bc4-f036-463f-949a-0fd9e18a4186 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:26.428405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fa5e6bc4-f036-463f-949a-0fd9e18a4186 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:26.436800 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa5e6bc4-f036-463f-949a-0fd9e18a4186 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:26.437832 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa5e6bc4-f036-463f-949a-0fd9e18a4186 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:26.438536 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa5e6bc4-f036-463f-949a-0fd9e18a4186 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:26.439484 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa5e6bc4-f036-463f-949a-0fd9e18a4186 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:26.440000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa5e6bc4-f036-463f-949a-0fd9e18a4186 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:26.440521 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa5e6bc4-f036-463f-949a-0fd9e18a4186 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:26.608111 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-fa5e6bc4-f036-463f-949a-0fd9e18a4186 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:26.620530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fa5e6bc4-f036-463f-949a-0fd9e18a4186 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.200646 Sep 08 13:57:26.623248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 213/430] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:26 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:26.705967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-34e796bb-6dac-4f96-b8ab-57b027e8b3a8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:26.707256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-34e796bb-6dac-4f96-b8ab-57b027e8b3a8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:26.715257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-34e796bb-6dac-4f96-b8ab-57b027e8b3a8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:26.716071 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-34e796bb-6dac-4f96-b8ab-57b027e8b3a8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:26.716724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-34e796bb-6dac-4f96-b8ab-57b027e8b3a8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:26.717884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-34e796bb-6dac-4f96-b8ab-57b027e8b3a8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:26.718748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-34e796bb-6dac-4f96-b8ab-57b027e8b3a8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:26.719177 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-34e796bb-6dac-4f96-b8ab-57b027e8b3a8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:26.911289 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-34e796bb-6dac-4f96-b8ab-57b027e8b3a8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:26.921709 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-34e796bb-6dac-4f96-b8ab-57b027e8b3a8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:26.924284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-34e796bb-6dac-4f96-b8ab-57b027e8b3a8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:26.925740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-34e796bb-6dac-4f96-b8ab-57b027e8b3a8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.224335 Sep 08 13:57:26.926518 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 218/431] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:26 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:27.648229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cb64c17c-4484-4bb4-b566-bd26bbd52497 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:27.649002 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cb64c17c-4484-4bb4-b566-bd26bbd52497 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:27.655971 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb64c17c-4484-4bb4-b566-bd26bbd52497 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:27.656529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb64c17c-4484-4bb4-b566-bd26bbd52497 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:27.656914 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb64c17c-4484-4bb4-b566-bd26bbd52497 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:27.657597 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb64c17c-4484-4bb4-b566-bd26bbd52497 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:27.658109 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb64c17c-4484-4bb4-b566-bd26bbd52497 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:27.667788 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb64c17c-4484-4bb4-b566-bd26bbd52497 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:27.824125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-cb64c17c-4484-4bb4-b566-bd26bbd52497 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:27.836252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cb64c17c-4484-4bb4-b566-bd26bbd52497 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.193213 Sep 08 13:57:27.850778 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 214/432] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:27 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:27.952010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e4914eb2-24cb-40f4-bbb5-81004141a015 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:27.952628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e4914eb2-24cb-40f4-bbb5-81004141a015 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:27.963973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e4914eb2-24cb-40f4-bbb5-81004141a015 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:27.964477 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e4914eb2-24cb-40f4-bbb5-81004141a015 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:27.964913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e4914eb2-24cb-40f4-bbb5-81004141a015 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:27.965886 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e4914eb2-24cb-40f4-bbb5-81004141a015 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:27.966390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e4914eb2-24cb-40f4-bbb5-81004141a015 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:27.966817 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e4914eb2-24cb-40f4-bbb5-81004141a015 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:28.129026 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e4914eb2-24cb-40f4-bbb5-81004141a015 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:28.140328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e4914eb2-24cb-40f4-bbb5-81004141a015 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:28.150544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e4914eb2-24cb-40f4-bbb5-81004141a015 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:28.150544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e4914eb2-24cb-40f4-bbb5-81004141a015 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.195295 Sep 08 13:57:28.150544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 219/433] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:27 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:28.860242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e8a91b81-18da-4bb5-bf63-17d4d6eac481 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:28.860966 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e8a91b81-18da-4bb5-bf63-17d4d6eac481 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:28.868512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8a91b81-18da-4bb5-bf63-17d4d6eac481 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:28.869003 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8a91b81-18da-4bb5-bf63-17d4d6eac481 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:28.869488 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8a91b81-18da-4bb5-bf63-17d4d6eac481 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:28.870216 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8a91b81-18da-4bb5-bf63-17d4d6eac481 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:28.870674 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8a91b81-18da-4bb5-bf63-17d4d6eac481 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:28.871167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8a91b81-18da-4bb5-bf63-17d4d6eac481 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:28.876225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:29.044904 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e8a91b81-18da-4bb5-bf63-17d4d6eac481 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:29.053121 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e8a91b81-18da-4bb5-bf63-17d4d6eac481 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.196696 Sep 08 13:57:29.068547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 215/434] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:28 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:29.195095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b2dcf619-12d7-415a-865b-8601b168c059 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:29.196194 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b2dcf619-12d7-415a-865b-8601b168c059 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:29.205824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b2dcf619-12d7-415a-865b-8601b168c059 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:29.206443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b2dcf619-12d7-415a-865b-8601b168c059 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:29.207032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b2dcf619-12d7-415a-865b-8601b168c059 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:29.207968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b2dcf619-12d7-415a-865b-8601b168c059 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:29.208424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b2dcf619-12d7-415a-865b-8601b168c059 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:29.209024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b2dcf619-12d7-415a-865b-8601b168c059 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:29.367814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b2dcf619-12d7-415a-865b-8601b168c059 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:29.383109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b2dcf619-12d7-415a-865b-8601b168c059 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:29.384597 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b2dcf619-12d7-415a-865b-8601b168c059 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:29.385771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b2dcf619-12d7-415a-865b-8601b168c059 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.194826 Sep 08 13:57:29.402540 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 220/435] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:29 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:30.078514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-13812f17-9158-4306-9dd0-f4b55b1ca5c9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:30.079271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-13812f17-9158-4306-9dd0-f4b55b1ca5c9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:30.086472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-13812f17-9158-4306-9dd0-f4b55b1ca5c9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:30.087350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-13812f17-9158-4306-9dd0-f4b55b1ca5c9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:30.088042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-13812f17-9158-4306-9dd0-f4b55b1ca5c9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:30.089022 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-13812f17-9158-4306-9dd0-f4b55b1ca5c9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:30.089710 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-13812f17-9158-4306-9dd0-f4b55b1ca5c9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:30.090275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-13812f17-9158-4306-9dd0-f4b55b1ca5c9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:30.091694 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0d8e8857-7816-4253-8978-a24fb2ec9488 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Action: 'create', calling method: .version_select at 0x7fce02240280>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-279563135", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBL9K3KJEaM8M1ZEVQyacwPQZUj0kXRbUcixurJkbwDkEIzucq6DLxFQChVAqlFik8FR312qwif+pdcKDvet4aFgPgj6D+z1fmMiHzuqSRB/cbKMFnB/xuGqbNzG1/tg7RQ=="}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:57:30.095208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:30.131009 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-0d8e8857-7816-4253-8978-a24fb2ec9488 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Getting quotas for project 179292c42f8e4041adc935c80dba0f70. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:57:30.136717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-0d8e8857-7816-4253-8978-a24fb2ec9488 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Getting quotas for user 1f10a95e49744a6dbb73e54e00d81add and project 179292c42f8e4041adc935c80dba0f70. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:57:30.216750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0d8e8857-7816-4253-8978-a24fb2ec9488 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.143955 Sep 08 13:57:30.222930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 216/436] 10.222.0.25 () {60 vars in 1159 bytes} [Thu Sep 8 13:57:30 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:57:30.271375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-13812f17-9158-4306-9dd0-f4b55b1ca5c9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:30.280931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-13812f17-9158-4306-9dd0-f4b55b1ca5c9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.207524 Sep 08 13:57:30.285833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 221/437] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:30 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:30.285973 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cbba9dd3-731b-41c9-bb78-637dcaaa19e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1238171246", "imageRef": "ce40814a-a7fa-41a9-9a8b-ff7c841a224a", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-279563135", "networks": [{"uuid": "3988684a-dd0f-4404-aacc-31f1bc1d6bf1"}]}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:57:30.364199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.network.neutron [None req-cbba9dd3-731b-41c9-bb78-637dcaaa19e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] validate_networks() for [('3988684a-dd0f-4404-aacc-31f1bc1d6bf1', None, None, None, None, None)] {{(pid=127602) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 13:57:30.412504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f0d312eb-749e-4144-840c-553e50d8453d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:30.414475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f0d312eb-749e-4144-840c-553e50d8453d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:30.424331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0d312eb-749e-4144-840c-553e50d8453d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:30.424331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0d312eb-749e-4144-840c-553e50d8453d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:30.424331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0d312eb-749e-4144-840c-553e50d8453d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:30.424331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0d312eb-749e-4144-840c-553e50d8453d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:30.424714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0d312eb-749e-4144-840c-553e50d8453d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:30.425088 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0d312eb-749e-4144-840c-553e50d8453d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:30.595645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f0d312eb-749e-4144-840c-553e50d8453d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:30.608736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f0d312eb-749e-4144-840c-553e50d8453d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:30.608736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f0d312eb-749e-4144-840c-553e50d8453d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:30.609122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f0d312eb-749e-4144-840c-553e50d8453d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.200298 Sep 08 13:57:30.609824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 222/438] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:30 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:30.631960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-ff2b2726-7070-4ab7-bec4-a615b7aa7a9d req-ce9ce01f-9baf-476a-a9ca-c824bb5f60e2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6c3f043c-0662-4b63-bbd5-3a0f68c89b94", "name": "network-vif-plugged", "status": "completed", "tag": "0476c717-e075-4f94-b8e8-3fa5f88138b1"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:57:30.654054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-ff2b2726-7070-4ab7-bec4-a615b7aa7a9d req-ce9ce01f-9baf-476a-a9ca-c824bb5f60e2 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:30.654586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-ff2b2726-7070-4ab7-bec4-a615b7aa7a9d req-ce9ce01f-9baf-476a-a9ca-c824bb5f60e2 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:30.655009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-ff2b2726-7070-4ab7-bec4-a615b7aa7a9d req-ce9ce01f-9baf-476a-a9ca-c824bb5f60e2 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:30.669022 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-ff2b2726-7070-4ab7-bec4-a615b7aa7a9d req-ce9ce01f-9baf-476a-a9ca-c824bb5f60e2 service nova] Creating event network-vif-plugged:0476c717-e075-4f94-b8e8-3fa5f88138b1 for instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 on hv-cinder-85646 Sep 08 13:57:30.674941 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-ff2b2726-7070-4ab7-bec4-a615b7aa7a9d req-ce9ce01f-9baf-476a-a9ca-c824bb5f60e2 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.063672 Sep 08 13:57:30.675507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 223/439] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:57:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:57:30.886748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-cbba9dd3-731b-41c9-bb78-637dcaaa19e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Flavor limits 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 13:57:30.887249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-cbba9dd3-731b-41c9-bb78-637dcaaa19e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Image limits 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 13:57:30.887815 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-cbba9dd3-731b-41c9-bb78-637dcaaa19e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Flavor pref 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 13:57:30.888497 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-cbba9dd3-731b-41c9-bb78-637dcaaa19e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Image pref 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 13:57:30.888984 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-cbba9dd3-731b-41c9-bb78-637dcaaa19e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 13:57:30.899755 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cbba9dd3-731b-41c9-bb78-637dcaaa19e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:30.900590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cbba9dd3-731b-41c9-bb78-637dcaaa19e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:30.901245 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cbba9dd3-731b-41c9-bb78-637dcaaa19e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:30.902503 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cbba9dd3-731b-41c9-bb78-637dcaaa19e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:30.903227 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cbba9dd3-731b-41c9-bb78-637dcaaa19e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:30.903861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cbba9dd3-731b-41c9-bb78-637dcaaa19e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:30.926386 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-cbba9dd3-731b-41c9-bb78-637dcaaa19e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Getting quotas for project 179292c42f8e4041adc935c80dba0f70. Resources: {'cores', 'instances', 'ram'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:57:30.931497 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-cbba9dd3-731b-41c9-bb78-637dcaaa19e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Getting quotas for user 1f10a95e49744a6dbb73e54e00d81add and project 179292c42f8e4041adc935c80dba0f70. Resources: {'cores', 'instances', 'ram'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:57:30.944817 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cbba9dd3-731b-41c9-bb78-637dcaaa19e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Going to run 1 instances... {{(pid=127602) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 13:57:30.947820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cbba9dd3-731b-41c9-bb78-637dcaaa19e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='ce40814a-a7fa-41a9-9a8b-ff7c841a224a',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127602) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 13:57:31.075954 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cbba9dd3-731b-41c9-bb78-637dcaaa19e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.797706 Sep 08 13:57:31.077157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 217/440] 10.222.0.25 () {60 vars in 1147 bytes} [Thu Sep 8 13:57:30 2022] POST /compute/v2.1/servers => generated 380 bytes in 800 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 08 13:57:31.095379 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9f5d355a-3bbf-4366-bf2a-5d8fe8e15256 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:31.097358 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9f5d355a-3bbf-4366-bf2a-5d8fe8e15256 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:31.233620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9f5d355a-3bbf-4366-bf2a-5d8fe8e15256 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:31.249357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9f5d355a-3bbf-4366-bf2a-5d8fe8e15256 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1258 microversion: 2.1 time: 0.162841 Sep 08 13:57:31.251721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 224/441] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:31 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1258 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:31.309011 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-88db22a3-61a3-41cc-81dd-273be700c587 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:31.309701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-88db22a3-61a3-41cc-81dd-273be700c587 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:31.319114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88db22a3-61a3-41cc-81dd-273be700c587 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:31.319987 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88db22a3-61a3-41cc-81dd-273be700c587 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:31.321360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88db22a3-61a3-41cc-81dd-273be700c587 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:31.322992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88db22a3-61a3-41cc-81dd-273be700c587 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:31.324013 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88db22a3-61a3-41cc-81dd-273be700c587 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:31.324610 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88db22a3-61a3-41cc-81dd-273be700c587 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:31.510909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-88db22a3-61a3-41cc-81dd-273be700c587 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:31.520681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-88db22a3-61a3-41cc-81dd-273be700c587 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.216141 Sep 08 13:57:31.537516 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 218/442] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:31 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:31.640732 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9ccbee09-b08d-4172-b8be-1e36a46190d4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:31.642198 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9ccbee09-b08d-4172-b8be-1e36a46190d4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:31.649123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9ccbee09-b08d-4172-b8be-1e36a46190d4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:31.653856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9ccbee09-b08d-4172-b8be-1e36a46190d4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:31.653856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9ccbee09-b08d-4172-b8be-1e36a46190d4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:31.653856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9ccbee09-b08d-4172-b8be-1e36a46190d4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:31.653856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9ccbee09-b08d-4172-b8be-1e36a46190d4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:31.654442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9ccbee09-b08d-4172-b8be-1e36a46190d4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:31.817440 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9ccbee09-b08d-4172-b8be-1e36a46190d4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:31.826876 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9ccbee09-b08d-4172-b8be-1e36a46190d4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:31.828556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9ccbee09-b08d-4172-b8be-1e36a46190d4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:31.829712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9ccbee09-b08d-4172-b8be-1e36a46190d4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.193535 Sep 08 13:57:31.830279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 225/443] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:31 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:32.286330 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-86e5cd75-98e1-4f65-82f6-a9c4a8f083ad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:32.286946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-86e5cd75-98e1-4f65-82f6-a9c4a8f083ad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:32.345134 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-86e5cd75-98e1-4f65-82f6-a9c4a8f083ad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:32.352964 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-86e5cd75-98e1-4f65-82f6-a9c4a8f083ad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1258 microversion: 2.1 time: 0.071520 Sep 08 13:57:32.353547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 219/444] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:32 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1258 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:32.544949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1303f876-581e-49ac-a596-64f0a204740e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:32.545570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1303f876-581e-49ac-a596-64f0a204740e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:32.552081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1303f876-581e-49ac-a596-64f0a204740e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:32.554077 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1303f876-581e-49ac-a596-64f0a204740e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:32.554077 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1303f876-581e-49ac-a596-64f0a204740e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:32.554077 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1303f876-581e-49ac-a596-64f0a204740e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:32.554444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1303f876-581e-49ac-a596-64f0a204740e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:32.554575 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1303f876-581e-49ac-a596-64f0a204740e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:32.700653 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1303f876-581e-49ac-a596-64f0a204740e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:32.708318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1303f876-581e-49ac-a596-64f0a204740e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.167400 Sep 08 13:57:32.708870 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 226/445] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:32 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:32.856615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4f9eef65-65e6-43ed-8035-4db83a37e377 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:32.857577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4f9eef65-65e6-43ed-8035-4db83a37e377 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:32.865033 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f9eef65-65e6-43ed-8035-4db83a37e377 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:32.865659 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f9eef65-65e6-43ed-8035-4db83a37e377 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:32.866235 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f9eef65-65e6-43ed-8035-4db83a37e377 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:32.867568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f9eef65-65e6-43ed-8035-4db83a37e377 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:32.868107 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f9eef65-65e6-43ed-8035-4db83a37e377 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:32.868530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f9eef65-65e6-43ed-8035-4db83a37e377 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:33.019574 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4f9eef65-65e6-43ed-8035-4db83a37e377 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:33.029561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4f9eef65-65e6-43ed-8035-4db83a37e377 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:33.031017 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4f9eef65-65e6-43ed-8035-4db83a37e377 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:33.032179 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4f9eef65-65e6-43ed-8035-4db83a37e377 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.181008 Sep 08 13:57:33.032741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 220/446] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:32 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:33.431189 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9a2b7f3e-1bcf-40aa-8a01-df94af18d52d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:33.431189 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9a2b7f3e-1bcf-40aa-8a01-df94af18d52d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:33.431189 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a2b7f3e-1bcf-40aa-8a01-df94af18d52d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:33.431189 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a2b7f3e-1bcf-40aa-8a01-df94af18d52d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:33.431542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a2b7f3e-1bcf-40aa-8a01-df94af18d52d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:33.431542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a2b7f3e-1bcf-40aa-8a01-df94af18d52d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:33.431542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a2b7f3e-1bcf-40aa-8a01-df94af18d52d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:33.431542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a2b7f3e-1bcf-40aa-8a01-df94af18d52d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:33.462105 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9a2b7f3e-1bcf-40aa-8a01-df94af18d52d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:33.469624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9a2b7f3e-1bcf-40aa-8a01-df94af18d52d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1262 microversion: 2.1 time: 0.097469 Sep 08 13:57:33.470379 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 227/447] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:33 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1262 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:33.733399 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-25522ea5-b87d-4e4a-93b0-b94ab40faaf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:33.734030 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-25522ea5-b87d-4e4a-93b0-b94ab40faaf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:33.741382 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-25522ea5-b87d-4e4a-93b0-b94ab40faaf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:33.741992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-25522ea5-b87d-4e4a-93b0-b94ab40faaf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:33.742534 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-25522ea5-b87d-4e4a-93b0-b94ab40faaf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:33.743308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-25522ea5-b87d-4e4a-93b0-b94ab40faaf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:33.743767 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-25522ea5-b87d-4e4a-93b0-b94ab40faaf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:33.744194 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-25522ea5-b87d-4e4a-93b0-b94ab40faaf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:33.893693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-25522ea5-b87d-4e4a-93b0-b94ab40faaf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:33.902959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-25522ea5-b87d-4e4a-93b0-b94ab40faaf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.173332 Sep 08 13:57:33.906001 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 221/448] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:33 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:34.058959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6e685e68-c5ef-4927-968f-cf5c432846e6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:34.060329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6e685e68-c5ef-4927-968f-cf5c432846e6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:34.068336 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6e685e68-c5ef-4927-968f-cf5c432846e6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:34.068834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6e685e68-c5ef-4927-968f-cf5c432846e6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:34.069484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6e685e68-c5ef-4927-968f-cf5c432846e6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:34.070736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6e685e68-c5ef-4927-968f-cf5c432846e6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:34.071196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6e685e68-c5ef-4927-968f-cf5c432846e6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:34.071831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6e685e68-c5ef-4927-968f-cf5c432846e6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:34.221949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6e685e68-c5ef-4927-968f-cf5c432846e6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:34.231104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6e685e68-c5ef-4927-968f-cf5c432846e6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:34.232777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6e685e68-c5ef-4927-968f-cf5c432846e6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:34.234200 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6e685e68-c5ef-4927-968f-cf5c432846e6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.179406 Sep 08 13:57:34.234762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 228/449] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:34 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:34.309792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b016469d-6bbc-4026-80fd-4d6e7be6a717 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3d6a13a3-142c-49d7-9ef4-0c61f151b4c3"}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:57:34.312202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b016469d-6bbc-4026-80fd-4d6e7be6a717 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:34.318394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b016469d-6bbc-4026-80fd-4d6e7be6a717 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:34.318874 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b016469d-6bbc-4026-80fd-4d6e7be6a717 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:34.319314 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b016469d-6bbc-4026-80fd-4d6e7be6a717 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:34.320579 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b016469d-6bbc-4026-80fd-4d6e7be6a717 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:34.321087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b016469d-6bbc-4026-80fd-4d6e7be6a717 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:34.321451 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b016469d-6bbc-4026-80fd-4d6e7be6a717 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:34.498443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2f29c061-2d1b-4057-88d0-60636ab4dbaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:34.503535 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2f29c061-2d1b-4057-88d0-60636ab4dbaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:34.510776 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f29c061-2d1b-4057-88d0-60636ab4dbaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:34.511390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f29c061-2d1b-4057-88d0-60636ab4dbaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:34.511769 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f29c061-2d1b-4057-88d0-60636ab4dbaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:34.512513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f29c061-2d1b-4057-88d0-60636ab4dbaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:34.512972 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f29c061-2d1b-4057-88d0-60636ab4dbaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:34.513401 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f29c061-2d1b-4057-88d0-60636ab4dbaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:34.581782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2f29c061-2d1b-4057-88d0-60636ab4dbaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:34.590215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2f29c061-2d1b-4057-88d0-60636ab4dbaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1262 microversion: 2.1 time: 0.096448 Sep 08 13:57:34.591105 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 229/450] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:34 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1262 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:34.928933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-895d4a70-7057-4e67-a3cd-503bba9be10d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:34.930206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-895d4a70-7057-4e67-a3cd-503bba9be10d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:34.936399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-895d4a70-7057-4e67-a3cd-503bba9be10d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:34.936898 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-895d4a70-7057-4e67-a3cd-503bba9be10d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:34.937337 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-895d4a70-7057-4e67-a3cd-503bba9be10d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:34.938052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-895d4a70-7057-4e67-a3cd-503bba9be10d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:34.938644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-895d4a70-7057-4e67-a3cd-503bba9be10d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:34.939078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-895d4a70-7057-4e67-a3cd-503bba9be10d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:35.113641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-895d4a70-7057-4e67-a3cd-503bba9be10d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:35.122089 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-895d4a70-7057-4e67-a3cd-503bba9be10d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.198979 Sep 08 13:57:35.122949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 230/451] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:34 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:35.259926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-84673c5e-c37c-4c8e-b110-3e166f2b5cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:35.260666 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-84673c5e-c37c-4c8e-b110-3e166f2b5cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:35.266610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84673c5e-c37c-4c8e-b110-3e166f2b5cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:35.267097 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84673c5e-c37c-4c8e-b110-3e166f2b5cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:35.267534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84673c5e-c37c-4c8e-b110-3e166f2b5cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:35.268219 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84673c5e-c37c-4c8e-b110-3e166f2b5cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:35.268673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84673c5e-c37c-4c8e-b110-3e166f2b5cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:35.269101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84673c5e-c37c-4c8e-b110-3e166f2b5cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:35.422186 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-84673c5e-c37c-4c8e-b110-3e166f2b5cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:35.431952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-84673c5e-c37c-4c8e-b110-3e166f2b5cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:35.433825 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-84673c5e-c37c-4c8e-b110-3e166f2b5cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:35.435161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-84673c5e-c37c-4c8e-b110-3e166f2b5cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.179127 Sep 08 13:57:35.435723 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 231/452] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:35 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:35.615483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-256adec9-b552-45cf-98fd-ffe4ed22a224 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:35.616565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-256adec9-b552-45cf-98fd-ffe4ed22a224 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:35.623194 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-256adec9-b552-45cf-98fd-ffe4ed22a224 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:35.623843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-256adec9-b552-45cf-98fd-ffe4ed22a224 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:35.624548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-256adec9-b552-45cf-98fd-ffe4ed22a224 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:35.625501 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-256adec9-b552-45cf-98fd-ffe4ed22a224 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:35.626211 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-256adec9-b552-45cf-98fd-ffe4ed22a224 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:35.626877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-256adec9-b552-45cf-98fd-ffe4ed22a224 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:35.704809 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-256adec9-b552-45cf-98fd-ffe4ed22a224 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:35.713239 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-256adec9-b552-45cf-98fd-ffe4ed22a224 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1262 microversion: 2.1 time: 0.102262 Sep 08 13:57:35.714117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 232/453] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:35 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1262 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:36.145666 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5a69dab0-87b1-4a24-b609-1b6f450bc4c3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:36.146383 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5a69dab0-87b1-4a24-b609-1b6f450bc4c3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:36.153460 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a69dab0-87b1-4a24-b609-1b6f450bc4c3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:36.160318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a69dab0-87b1-4a24-b609-1b6f450bc4c3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:36.160318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a69dab0-87b1-4a24-b609-1b6f450bc4c3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:36.160318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a69dab0-87b1-4a24-b609-1b6f450bc4c3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:36.160318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a69dab0-87b1-4a24-b609-1b6f450bc4c3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:36.160989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a69dab0-87b1-4a24-b609-1b6f450bc4c3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:36.301944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5a69dab0-87b1-4a24-b609-1b6f450bc4c3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:36.310836 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5a69dab0-87b1-4a24-b609-1b6f450bc4c3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.169075 Sep 08 13:57:36.311666 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 233/454] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:36 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:36.462911 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c578ab55-f53d-42de-bb86-de2f9ee34fd8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:36.463519 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c578ab55-f53d-42de-bb86-de2f9ee34fd8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:36.469532 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c578ab55-f53d-42de-bb86-de2f9ee34fd8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:36.470180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c578ab55-f53d-42de-bb86-de2f9ee34fd8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:36.470180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c578ab55-f53d-42de-bb86-de2f9ee34fd8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:36.471286 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c578ab55-f53d-42de-bb86-de2f9ee34fd8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:36.471917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c578ab55-f53d-42de-bb86-de2f9ee34fd8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:36.472165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c578ab55-f53d-42de-bb86-de2f9ee34fd8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:36.672715 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c578ab55-f53d-42de-bb86-de2f9ee34fd8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:36.672715 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c578ab55-f53d-42de-bb86-de2f9ee34fd8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:36.673789 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c578ab55-f53d-42de-bb86-de2f9ee34fd8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:36.673789 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c578ab55-f53d-42de-bb86-de2f9ee34fd8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.209820 Sep 08 13:57:36.673789 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 234/455] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:36 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:36.737931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-975dcab2-d0d6-4e64-83f8-19d7c82e2007 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:36.738576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-975dcab2-d0d6-4e64-83f8-19d7c82e2007 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:36.745667 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-975dcab2-d0d6-4e64-83f8-19d7c82e2007 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:36.746088 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-975dcab2-d0d6-4e64-83f8-19d7c82e2007 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:36.746562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-975dcab2-d0d6-4e64-83f8-19d7c82e2007 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:36.747547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-975dcab2-d0d6-4e64-83f8-19d7c82e2007 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:36.748021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-975dcab2-d0d6-4e64-83f8-19d7c82e2007 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:36.748447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-975dcab2-d0d6-4e64-83f8-19d7c82e2007 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:36.816007 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-975dcab2-d0d6-4e64-83f8-19d7c82e2007 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:36.825448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-975dcab2-d0d6-4e64-83f8-19d7c82e2007 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1262 microversion: 2.1 time: 0.092548 Sep 08 13:57:36.826031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 235/456] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:36 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1262 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:37.335262 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fe7b6b55-67bc-4aa5-8154-f4f39250a461 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:37.336867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fe7b6b55-67bc-4aa5-8154-f4f39250a461 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:37.346396 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fe7b6b55-67bc-4aa5-8154-f4f39250a461 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:37.347259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fe7b6b55-67bc-4aa5-8154-f4f39250a461 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:37.347865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fe7b6b55-67bc-4aa5-8154-f4f39250a461 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:37.348818 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fe7b6b55-67bc-4aa5-8154-f4f39250a461 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:37.349607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fe7b6b55-67bc-4aa5-8154-f4f39250a461 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:37.350256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fe7b6b55-67bc-4aa5-8154-f4f39250a461 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:37.549413 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fe7b6b55-67bc-4aa5-8154-f4f39250a461 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:37.559088 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fe7b6b55-67bc-4aa5-8154-f4f39250a461 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.228138 Sep 08 13:57:37.560225 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 236/457] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:37 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:37.694107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6a354ea6-1890-4fe0-939c-4c3e384cfd33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:37.695045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6a354ea6-1890-4fe0-939c-4c3e384cfd33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:37.701621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a354ea6-1890-4fe0-939c-4c3e384cfd33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:37.702129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a354ea6-1890-4fe0-939c-4c3e384cfd33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:37.702622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a354ea6-1890-4fe0-939c-4c3e384cfd33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:37.703312 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a354ea6-1890-4fe0-939c-4c3e384cfd33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:37.703762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a354ea6-1890-4fe0-939c-4c3e384cfd33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:37.704229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a354ea6-1890-4fe0-939c-4c3e384cfd33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:37.872750 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6a354ea6-1890-4fe0-939c-4c3e384cfd33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:37.882332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6a354ea6-1890-4fe0-939c-4c3e384cfd33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:37.883798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6a354ea6-1890-4fe0-939c-4c3e384cfd33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:37.884997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6a354ea6-1890-4fe0-939c-4c3e384cfd33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.195735 Sep 08 13:57:37.885556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 237/458] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:37 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:37.890028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b788621d-be66-4971-8c35-b1785d0ebe11 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:37.890728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b788621d-be66-4971-8c35-b1785d0ebe11 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:37.896773 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b788621d-be66-4971-8c35-b1785d0ebe11 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:37.897250 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b788621d-be66-4971-8c35-b1785d0ebe11 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:37.897680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b788621d-be66-4971-8c35-b1785d0ebe11 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:37.898407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b788621d-be66-4971-8c35-b1785d0ebe11 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:37.898851 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b788621d-be66-4971-8c35-b1785d0ebe11 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:37.899268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b788621d-be66-4971-8c35-b1785d0ebe11 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:37.963423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b788621d-be66-4971-8c35-b1785d0ebe11 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:37.971785 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b788621d-be66-4971-8c35-b1785d0ebe11 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1262 microversion: 2.1 time: 0.085184 Sep 08 13:57:37.972359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 238/459] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:37 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1262 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:38.584034 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c1035b39-586e-451f-b200-42b0bbcc7af3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:38.584846 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c1035b39-586e-451f-b200-42b0bbcc7af3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:38.592720 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1035b39-586e-451f-b200-42b0bbcc7af3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:38.593324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1035b39-586e-451f-b200-42b0bbcc7af3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:38.593867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1035b39-586e-451f-b200-42b0bbcc7af3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:38.594789 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1035b39-586e-451f-b200-42b0bbcc7af3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:38.595308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1035b39-586e-451f-b200-42b0bbcc7af3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:38.595791 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1035b39-586e-451f-b200-42b0bbcc7af3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:38.614499 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:39.009337 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c1035b39-586e-451f-b200-42b0bbcc7af3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:39.016860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c1035b39-586e-451f-b200-42b0bbcc7af3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.436530 Sep 08 13:57:39.017634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 239/460] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:38 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 438 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:39.022348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8f300509-6610-49ea-8d93-e02d5dd027b8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:39.022930 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8f300509-6610-49ea-8d93-e02d5dd027b8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:39.030759 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8f300509-6610-49ea-8d93-e02d5dd027b8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:39.031318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8f300509-6610-49ea-8d93-e02d5dd027b8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:39.031842 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8f300509-6610-49ea-8d93-e02d5dd027b8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:39.032569 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8f300509-6610-49ea-8d93-e02d5dd027b8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:39.033044 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8f300509-6610-49ea-8d93-e02d5dd027b8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:39.033609 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8f300509-6610-49ea-8d93-e02d5dd027b8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:39.194011 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8f300509-6610-49ea-8d93-e02d5dd027b8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:39.204463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8f300509-6610-49ea-8d93-e02d5dd027b8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:39.205898 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8f300509-6610-49ea-8d93-e02d5dd027b8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:39.207288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8f300509-6610-49ea-8d93-e02d5dd027b8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.188909 Sep 08 13:57:39.207858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 240/461] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:39 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:39.213333 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f0bee80f-8361-48b3-8b46-d6eaf8a05511 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:39.213932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f0bee80f-8361-48b3-8b46-d6eaf8a05511 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:39.220275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0bee80f-8361-48b3-8b46-d6eaf8a05511 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:39.220892 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0bee80f-8361-48b3-8b46-d6eaf8a05511 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:39.221326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0bee80f-8361-48b3-8b46-d6eaf8a05511 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:39.222031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0bee80f-8361-48b3-8b46-d6eaf8a05511 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:39.222512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0bee80f-8361-48b3-8b46-d6eaf8a05511 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:39.222929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0bee80f-8361-48b3-8b46-d6eaf8a05511 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:39.300778 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f0bee80f-8361-48b3-8b46-d6eaf8a05511 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:39.309391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f0bee80f-8361-48b3-8b46-d6eaf8a05511 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1262 microversion: 2.1 time: 0.100249 Sep 08 13:57:39.310647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 241/462] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:39 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1262 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:40.042237 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a7ee2c1d-1367-40bc-b6c6-fbb89c3fccb8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:40.042902 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a7ee2c1d-1367-40bc-b6c6-fbb89c3fccb8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:40.049647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7ee2c1d-1367-40bc-b6c6-fbb89c3fccb8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:40.050171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7ee2c1d-1367-40bc-b6c6-fbb89c3fccb8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:40.050602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7ee2c1d-1367-40bc-b6c6-fbb89c3fccb8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:40.051308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7ee2c1d-1367-40bc-b6c6-fbb89c3fccb8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:40.051945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7ee2c1d-1367-40bc-b6c6-fbb89c3fccb8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:40.052532 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7ee2c1d-1367-40bc-b6c6-fbb89c3fccb8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:40.208860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a7ee2c1d-1367-40bc-b6c6-fbb89c3fccb8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:40.222034 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a7ee2c1d-1367-40bc-b6c6-fbb89c3fccb8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.182526 Sep 08 13:57:40.224118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 242/463] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:40 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:40.236341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c5337028-1131-4591-99ee-cb7b2152ebba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:40.236341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c5337028-1131-4591-99ee-cb7b2152ebba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:40.251353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5337028-1131-4591-99ee-cb7b2152ebba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:40.251353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5337028-1131-4591-99ee-cb7b2152ebba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:40.251353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5337028-1131-4591-99ee-cb7b2152ebba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:40.251353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5337028-1131-4591-99ee-cb7b2152ebba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:40.252033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5337028-1131-4591-99ee-cb7b2152ebba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:40.252688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5337028-1131-4591-99ee-cb7b2152ebba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:40.429184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c5337028-1131-4591-99ee-cb7b2152ebba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:40.436685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c5337028-1131-4591-99ee-cb7b2152ebba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:40.438248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c5337028-1131-4591-99ee-cb7b2152ebba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:40.439408 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c5337028-1131-4591-99ee-cb7b2152ebba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.208926 Sep 08 13:57:40.439967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 243/464] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:40 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:40.445247 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2ca3ee3d-7069-4e56-97aa-2c773ef05dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:40.445899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2ca3ee3d-7069-4e56-97aa-2c773ef05dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:40.452327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ca3ee3d-7069-4e56-97aa-2c773ef05dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:40.458061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ca3ee3d-7069-4e56-97aa-2c773ef05dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:40.458061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ca3ee3d-7069-4e56-97aa-2c773ef05dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:40.458061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ca3ee3d-7069-4e56-97aa-2c773ef05dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:40.458061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ca3ee3d-7069-4e56-97aa-2c773ef05dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:40.458468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ca3ee3d-7069-4e56-97aa-2c773ef05dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:40.523852 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2ca3ee3d-7069-4e56-97aa-2c773ef05dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:40.533159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2ca3ee3d-7069-4e56-97aa-2c773ef05dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1262 microversion: 2.1 time: 0.092077 Sep 08 13:57:40.547126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 244/465] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:40 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1262 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:40.917324 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=2 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:41.246950 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a61a1955-6bd9-44e3-a587-52b5c5750417 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:41.247776 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a61a1955-6bd9-44e3-a587-52b5c5750417 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:41.250100 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=2 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:41.255379 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a61a1955-6bd9-44e3-a587-52b5c5750417 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:41.256008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a61a1955-6bd9-44e3-a587-52b5c5750417 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:41.256411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a61a1955-6bd9-44e3-a587-52b5c5750417 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:41.257091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a61a1955-6bd9-44e3-a587-52b5c5750417 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:41.257541 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a61a1955-6bd9-44e3-a587-52b5c5750417 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:41.257963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a61a1955-6bd9-44e3-a587-52b5c5750417 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:41.407774 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a61a1955-6bd9-44e3-a587-52b5c5750417 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:41.416931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a61a1955-6bd9-44e3-a587-52b5c5750417 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.173997 Sep 08 13:57:41.417873 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 245/466] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:41 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:41.465764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-841053ab-2a94-44d1-b5f0-e2a16e631d41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:41.466326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-841053ab-2a94-44d1-b5f0-e2a16e631d41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:41.473082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-841053ab-2a94-44d1-b5f0-e2a16e631d41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:41.474445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-841053ab-2a94-44d1-b5f0-e2a16e631d41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:41.474945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-841053ab-2a94-44d1-b5f0-e2a16e631d41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:41.475673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-841053ab-2a94-44d1-b5f0-e2a16e631d41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:41.476175 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-841053ab-2a94-44d1-b5f0-e2a16e631d41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:41.476624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-841053ab-2a94-44d1-b5f0-e2a16e631d41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:41.626679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-841053ab-2a94-44d1-b5f0-e2a16e631d41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:41.637408 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-841053ab-2a94-44d1-b5f0-e2a16e631d41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:41.643632 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-841053ab-2a94-44d1-b5f0-e2a16e631d41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:41.643632 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-841053ab-2a94-44d1-b5f0-e2a16e631d41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.178812 Sep 08 13:57:41.643632 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 246/467] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:41 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:41.645131 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d6821318-3a40-4af9-b0e2-d675fa2b4de8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:41.645757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d6821318-3a40-4af9-b0e2-d675fa2b4de8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:41.652408 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d6821318-3a40-4af9-b0e2-d675fa2b4de8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:41.653141 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d6821318-3a40-4af9-b0e2-d675fa2b4de8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:41.653573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d6821318-3a40-4af9-b0e2-d675fa2b4de8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:41.654296 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d6821318-3a40-4af9-b0e2-d675fa2b4de8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:41.654753 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d6821318-3a40-4af9-b0e2-d675fa2b4de8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:41.655224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d6821318-3a40-4af9-b0e2-d675fa2b4de8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:41.725606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d6821318-3a40-4af9-b0e2-d675fa2b4de8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:41.733546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d6821318-3a40-4af9-b0e2-d675fa2b4de8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1262 microversion: 2.1 time: 0.092383 Sep 08 13:57:41.734113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 247/468] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:41 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1262 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:42.349162 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=12 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:42.440566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9e711007-25d4-4ec0-913d-549e66f2d54b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:42.441167 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9e711007-25d4-4ec0-913d-549e66f2d54b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:42.447832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e711007-25d4-4ec0-913d-549e66f2d54b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:42.448322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e711007-25d4-4ec0-913d-549e66f2d54b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:42.448763 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e711007-25d4-4ec0-913d-549e66f2d54b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:42.449588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e711007-25d4-4ec0-913d-549e66f2d54b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:42.450205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e711007-25d4-4ec0-913d-549e66f2d54b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:42.450657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e711007-25d4-4ec0-913d-549e66f2d54b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:42.591649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9e711007-25d4-4ec0-913d-549e66f2d54b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:42.600318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9e711007-25d4-4ec0-913d-549e66f2d54b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.163600 Sep 08 13:57:42.606255 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 248/469] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:42 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:42.666714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-458f2add-c5ae-4909-9e73-f0c904c41001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:42.667434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-458f2add-c5ae-4909-9e73-f0c904c41001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:42.673682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-458f2add-c5ae-4909-9e73-f0c904c41001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:42.674204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-458f2add-c5ae-4909-9e73-f0c904c41001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:42.674651 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-458f2add-c5ae-4909-9e73-f0c904c41001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:42.675360 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-458f2add-c5ae-4909-9e73-f0c904c41001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:42.675819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-458f2add-c5ae-4909-9e73-f0c904c41001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:42.676249 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-458f2add-c5ae-4909-9e73-f0c904c41001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:42.817969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-458f2add-c5ae-4909-9e73-f0c904c41001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:42.828586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-458f2add-c5ae-4909-9e73-f0c904c41001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:42.830082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-458f2add-c5ae-4909-9e73-f0c904c41001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:42.831266 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-458f2add-c5ae-4909-9e73-f0c904c41001 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.169339 Sep 08 13:57:42.831846 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 249/470] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:42 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:42.836358 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-281fddc4-b4f1-42ee-8d8c-8bb5e1d201fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:42.837025 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-281fddc4-b4f1-42ee-8d8c-8bb5e1d201fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:42.842805 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-281fddc4-b4f1-42ee-8d8c-8bb5e1d201fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:42.843341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-281fddc4-b4f1-42ee-8d8c-8bb5e1d201fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:42.843771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-281fddc4-b4f1-42ee-8d8c-8bb5e1d201fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:42.844468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-281fddc4-b4f1-42ee-8d8c-8bb5e1d201fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:42.844913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-281fddc4-b4f1-42ee-8d8c-8bb5e1d201fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:42.845332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-281fddc4-b4f1-42ee-8d8c-8bb5e1d201fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:42.914533 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-281fddc4-b4f1-42ee-8d8c-8bb5e1d201fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:42.922303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-281fddc4-b4f1-42ee-8d8c-8bb5e1d201fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1254 microversion: 2.1 time: 0.089388 Sep 08 13:57:42.922941 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 250/471] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:42 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1254 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:43.624618 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-89d7fd43-fd97-4090-989f-430c51f3c8e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:43.625312 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-89d7fd43-fd97-4090-989f-430c51f3c8e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:43.631835 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-89d7fd43-fd97-4090-989f-430c51f3c8e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:43.632330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-89d7fd43-fd97-4090-989f-430c51f3c8e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:43.632771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-89d7fd43-fd97-4090-989f-430c51f3c8e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:43.634006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-89d7fd43-fd97-4090-989f-430c51f3c8e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:43.634499 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-89d7fd43-fd97-4090-989f-430c51f3c8e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:43.634934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-89d7fd43-fd97-4090-989f-430c51f3c8e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:43.779857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-89d7fd43-fd97-4090-989f-430c51f3c8e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:43.788135 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-89d7fd43-fd97-4090-989f-430c51f3c8e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.167354 Sep 08 13:57:43.789005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 251/472] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:43 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:43.862261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8ab69d75-cd3c-4985-8565-232049134abe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:43.863470 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8ab69d75-cd3c-4985-8565-232049134abe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:43.873574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8ab69d75-cd3c-4985-8565-232049134abe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:43.873574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8ab69d75-cd3c-4985-8565-232049134abe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:43.873574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8ab69d75-cd3c-4985-8565-232049134abe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:43.873574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8ab69d75-cd3c-4985-8565-232049134abe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:43.873982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8ab69d75-cd3c-4985-8565-232049134abe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:43.873982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8ab69d75-cd3c-4985-8565-232049134abe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:44.041095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8ab69d75-cd3c-4985-8565-232049134abe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:44.050636 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8ab69d75-cd3c-4985-8565-232049134abe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:44.052078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8ab69d75-cd3c-4985-8565-232049134abe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:44.053263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8ab69d75-cd3c-4985-8565-232049134abe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.195284 Sep 08 13:57:44.053818 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 252/473] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:43 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:44.059408 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a76cdb9c-9e8d-46bb-b0a0-0f587240464b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:44.060079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a76cdb9c-9e8d-46bb-b0a0-0f587240464b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:44.066393 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a76cdb9c-9e8d-46bb-b0a0-0f587240464b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:44.066893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a76cdb9c-9e8d-46bb-b0a0-0f587240464b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:44.067332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a76cdb9c-9e8d-46bb-b0a0-0f587240464b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:44.068062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a76cdb9c-9e8d-46bb-b0a0-0f587240464b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:44.068494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a76cdb9c-9e8d-46bb-b0a0-0f587240464b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:44.068917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a76cdb9c-9e8d-46bb-b0a0-0f587240464b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:44.137453 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a76cdb9c-9e8d-46bb-b0a0-0f587240464b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:44.146246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a76cdb9c-9e8d-46bb-b0a0-0f587240464b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1310 microversion: 2.1 time: 0.091299 Sep 08 13:57:44.146829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 253/474] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:44 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1310 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:44.315970 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=65,nova_api:INSERT=4 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:44.456228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b016469d-6bbc-4026-80fd-4d6e7be6a717 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "POST /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 10.150256 Sep 08 13:57:44.456863 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 222/475] 10.222.0.25 () {60 vars in 1323 bytes} [Thu Sep 8 13:57:34 2022] POST /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 194 bytes in 10152 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:57:44.812484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a7bcdb12-8a81-401b-9d44-3479aa3ebbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:44.813054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a7bcdb12-8a81-401b-9d44-3479aa3ebbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:44.819733 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7bcdb12-8a81-401b-9d44-3479aa3ebbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:44.820233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7bcdb12-8a81-401b-9d44-3479aa3ebbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:44.820680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7bcdb12-8a81-401b-9d44-3479aa3ebbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:44.821418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7bcdb12-8a81-401b-9d44-3479aa3ebbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:44.821874 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7bcdb12-8a81-401b-9d44-3479aa3ebbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:44.822337 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7bcdb12-8a81-401b-9d44-3479aa3ebbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:44.961276 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a7bcdb12-8a81-401b-9d44-3479aa3ebbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:44.969883 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a7bcdb12-8a81-401b-9d44-3479aa3ebbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.161378 Sep 08 13:57:44.970500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 254/476] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:44 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:45.081378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c25b060b-6196-4f33-81ab-28897c9fe3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:45.082208 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c25b060b-6196-4f33-81ab-28897c9fe3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:45.103566 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c25b060b-6196-4f33-81ab-28897c9fe3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:45.104057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c25b060b-6196-4f33-81ab-28897c9fe3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:45.106842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c25b060b-6196-4f33-81ab-28897c9fe3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:45.106842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c25b060b-6196-4f33-81ab-28897c9fe3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:45.106842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c25b060b-6196-4f33-81ab-28897c9fe3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:45.106842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c25b060b-6196-4f33-81ab-28897c9fe3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:45.172422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ba05c6d9-53d5-490c-a0d6-9cc311585b4b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:45.173122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ba05c6d9-53d5-490c-a0d6-9cc311585b4b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:45.180065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba05c6d9-53d5-490c-a0d6-9cc311585b4b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:45.182027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba05c6d9-53d5-490c-a0d6-9cc311585b4b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:45.182584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba05c6d9-53d5-490c-a0d6-9cc311585b4b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:45.183394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba05c6d9-53d5-490c-a0d6-9cc311585b4b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:45.183932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba05c6d9-53d5-490c-a0d6-9cc311585b4b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:45.184438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba05c6d9-53d5-490c-a0d6-9cc311585b4b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:45.262234 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ba05c6d9-53d5-490c-a0d6-9cc311585b4b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:45.271455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ba05c6d9-53d5-490c-a0d6-9cc311585b4b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1310 microversion: 2.1 time: 0.103204 Sep 08 13:57:45.272181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 255/477] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:45 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1310 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:45.357806 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c25b060b-6196-4f33-81ab-28897c9fe3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:45.369097 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c25b060b-6196-4f33-81ab-28897c9fe3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:45.370857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c25b060b-6196-4f33-81ab-28897c9fe3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:45.372127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c25b060b-6196-4f33-81ab-28897c9fe3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.295798 Sep 08 13:57:45.372712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 223/478] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:45 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:45.993485 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a0e9ed63-0681-4772-a68a-4c95a6213ad6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:45.994115 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a0e9ed63-0681-4772-a68a-4c95a6213ad6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:46.001033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a0e9ed63-0681-4772-a68a-4c95a6213ad6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:46.001520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a0e9ed63-0681-4772-a68a-4c95a6213ad6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:46.001955 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a0e9ed63-0681-4772-a68a-4c95a6213ad6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:46.002697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a0e9ed63-0681-4772-a68a-4c95a6213ad6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:46.003203 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a0e9ed63-0681-4772-a68a-4c95a6213ad6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:46.003620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a0e9ed63-0681-4772-a68a-4c95a6213ad6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:46.006769 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:46.148023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a0e9ed63-0681-4772-a68a-4c95a6213ad6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:46.157868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a0e9ed63-0681-4772-a68a-4c95a6213ad6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.168836 Sep 08 13:57:46.158476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 256/479] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:45 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:46.295307 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-12bfd080-27d5-4a57-8dc9-2c696de81f9b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:46.295831 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-12bfd080-27d5-4a57-8dc9-2c696de81f9b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:46.302479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-12bfd080-27d5-4a57-8dc9-2c696de81f9b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:46.302965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-12bfd080-27d5-4a57-8dc9-2c696de81f9b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:46.303392 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-12bfd080-27d5-4a57-8dc9-2c696de81f9b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:46.304086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-12bfd080-27d5-4a57-8dc9-2c696de81f9b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:46.304531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-12bfd080-27d5-4a57-8dc9-2c696de81f9b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:46.304950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-12bfd080-27d5-4a57-8dc9-2c696de81f9b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:46.368355 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-12bfd080-27d5-4a57-8dc9-2c696de81f9b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:46.376074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-12bfd080-27d5-4a57-8dc9-2c696de81f9b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1310 microversion: 2.1 time: 0.084797 Sep 08 13:57:46.376681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 224/480] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:46 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1310 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:46.399030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1cde112a-3e75-4ca9-9b10-abbc2b3c2cc9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:46.399633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1cde112a-3e75-4ca9-9b10-abbc2b3c2cc9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:46.405813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1cde112a-3e75-4ca9-9b10-abbc2b3c2cc9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:46.406419 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1cde112a-3e75-4ca9-9b10-abbc2b3c2cc9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:46.406853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1cde112a-3e75-4ca9-9b10-abbc2b3c2cc9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:46.407737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1cde112a-3e75-4ca9-9b10-abbc2b3c2cc9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:46.408373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1cde112a-3e75-4ca9-9b10-abbc2b3c2cc9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:46.408802 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1cde112a-3e75-4ca9-9b10-abbc2b3c2cc9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:46.548578 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1cde112a-3e75-4ca9-9b10-abbc2b3c2cc9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:46.559129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1cde112a-3e75-4ca9-9b10-abbc2b3c2cc9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:46.564221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1cde112a-3e75-4ca9-9b10-abbc2b3c2cc9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:46.564221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1cde112a-3e75-4ca9-9b10-abbc2b3c2cc9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.166809 Sep 08 13:57:46.564221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 257/481] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:46 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:47.180294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c33ae65e-9d7f-4586-8983-dd8938672744 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:47.181984 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c33ae65e-9d7f-4586-8983-dd8938672744 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:47.188458 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c33ae65e-9d7f-4586-8983-dd8938672744 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:47.188948 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c33ae65e-9d7f-4586-8983-dd8938672744 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:47.189382 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c33ae65e-9d7f-4586-8983-dd8938672744 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:47.190084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c33ae65e-9d7f-4586-8983-dd8938672744 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:47.190581 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c33ae65e-9d7f-4586-8983-dd8938672744 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:47.191008 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c33ae65e-9d7f-4586-8983-dd8938672744 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:47.329657 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c33ae65e-9d7f-4586-8983-dd8938672744 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:47.339458 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c33ae65e-9d7f-4586-8983-dd8938672744 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1264 microversion: 2.1 time: 0.162984 Sep 08 13:57:47.340233 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 225/482] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:47 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1264 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:47.401811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-530a0808-8579-47a4-9b32-ca14b5469f8d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:47.402432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-530a0808-8579-47a4-9b32-ca14b5469f8d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:47.409131 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-530a0808-8579-47a4-9b32-ca14b5469f8d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:47.409621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-530a0808-8579-47a4-9b32-ca14b5469f8d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:47.410065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-530a0808-8579-47a4-9b32-ca14b5469f8d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:47.410785 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-530a0808-8579-47a4-9b32-ca14b5469f8d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:47.411246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-530a0808-8579-47a4-9b32-ca14b5469f8d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:47.411706 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-530a0808-8579-47a4-9b32-ca14b5469f8d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:47.473042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-530a0808-8579-47a4-9b32-ca14b5469f8d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:47.481326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-530a0808-8579-47a4-9b32-ca14b5469f8d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1328 microversion: 2.1 time: 0.083391 Sep 08 13:57:47.481892 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 258/483] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:47 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1328 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:47.588453 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-286ab252-3a81-4a38-affa-fba9b49be434 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:47.589160 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-286ab252-3a81-4a38-affa-fba9b49be434 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:47.599674 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-286ab252-3a81-4a38-affa-fba9b49be434 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:47.599674 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-286ab252-3a81-4a38-affa-fba9b49be434 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:47.599979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-286ab252-3a81-4a38-affa-fba9b49be434 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:47.600465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-286ab252-3a81-4a38-affa-fba9b49be434 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:47.600922 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-286ab252-3a81-4a38-affa-fba9b49be434 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:47.601376 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-286ab252-3a81-4a38-affa-fba9b49be434 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:47.785049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-286ab252-3a81-4a38-affa-fba9b49be434 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:47.794612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-286ab252-3a81-4a38-affa-fba9b49be434 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:47.796049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-286ab252-3a81-4a38-affa-fba9b49be434 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:47.797204 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-286ab252-3a81-4a38-affa-fba9b49be434 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.212734 Sep 08 13:57:47.797762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 226/484] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:47 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:48.363903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cc12deab-49fd-4ae2-a330-b9de392387fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:48.364488 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cc12deab-49fd-4ae2-a330-b9de392387fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:48.372026 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cc12deab-49fd-4ae2-a330-b9de392387fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:48.372514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cc12deab-49fd-4ae2-a330-b9de392387fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:48.373352 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cc12deab-49fd-4ae2-a330-b9de392387fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:48.374952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cc12deab-49fd-4ae2-a330-b9de392387fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:48.375663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cc12deab-49fd-4ae2-a330-b9de392387fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:48.376104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cc12deab-49fd-4ae2-a330-b9de392387fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:48.505104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0141a26e-e8bf-4789-8132-04b009954000 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:48.505824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0141a26e-e8bf-4789-8132-04b009954000 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:48.512966 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0141a26e-e8bf-4789-8132-04b009954000 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:48.513446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0141a26e-e8bf-4789-8132-04b009954000 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:48.514369 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0141a26e-e8bf-4789-8132-04b009954000 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:48.522868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cc12deab-49fd-4ae2-a330-b9de392387fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:48.532452 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0141a26e-e8bf-4789-8132-04b009954000 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:48.532950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0141a26e-e8bf-4789-8132-04b009954000 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:48.533397 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0141a26e-e8bf-4789-8132-04b009954000 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:48.538572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cc12deab-49fd-4ae2-a330-b9de392387fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1450 microversion: 2.1 time: 0.178691 Sep 08 13:57:48.540905 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 259/485] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:48 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1450 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:48.563963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f25dccd7-7c93-4a3d-81c5-4edc78cf89e9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:48.564571 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f25dccd7-7c93-4a3d-81c5-4edc78cf89e9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:48.571794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f25dccd7-7c93-4a3d-81c5-4edc78cf89e9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:48.572210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f25dccd7-7c93-4a3d-81c5-4edc78cf89e9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:48.572647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f25dccd7-7c93-4a3d-81c5-4edc78cf89e9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:48.573341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f25dccd7-7c93-4a3d-81c5-4edc78cf89e9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:48.573789 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f25dccd7-7c93-4a3d-81c5-4edc78cf89e9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:48.574417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f25dccd7-7c93-4a3d-81c5-4edc78cf89e9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:48.683380 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0141a26e-e8bf-4789-8132-04b009954000 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:48.692118 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0141a26e-e8bf-4789-8132-04b009954000 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.190999 Sep 08 13:57:48.693082 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 227/486] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:48 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:48.722302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f25dccd7-7c93-4a3d-81c5-4edc78cf89e9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:48.729815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f25dccd7-7c93-4a3d-81c5-4edc78cf89e9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1450 microversion: 2.1 time: 0.169679 Sep 08 13:57:48.730553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 260/487] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:48 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1450 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:48.823910 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c1a32907-4340-4ecc-85c9-1421c131b442 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:48.824513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c1a32907-4340-4ecc-85c9-1421c131b442 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:48.833670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c1a32907-4340-4ecc-85c9-1421c131b442 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:48.834663 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c1a32907-4340-4ecc-85c9-1421c131b442 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:48.835340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c1a32907-4340-4ecc-85c9-1421c131b442 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:48.836255 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c1a32907-4340-4ecc-85c9-1421c131b442 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:48.838043 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c1a32907-4340-4ecc-85c9-1421c131b442 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:48.838954 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c1a32907-4340-4ecc-85c9-1421c131b442 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:49.109311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c1a32907-4340-4ecc-85c9-1421c131b442 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:49.125045 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c1a32907-4340-4ecc-85c9-1421c131b442 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:49.127197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c1a32907-4340-4ecc-85c9-1421c131b442 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:49.129387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c1a32907-4340-4ecc-85c9-1421c131b442 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.309526 Sep 08 13:57:49.130613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 228/488] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:48 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:49.462243 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8a70ea4f-2275-4cdc-a55a-fedc28be725e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-492654622", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1782896132", "security_groups": [{"name": "tempest-secgroup-smoke-1656704105"}], "networks": [{"uuid": "869f7783-15f6-4d39-b589-230936c9035a"}], "block_device_mapping_v2": [{"uuid": "2012ff93-344d-4a13-879f-779050ac0a2d", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:57:49.657853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.network.neutron [None req-8a70ea4f-2275-4cdc-a55a-fedc28be725e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] validate_networks() for [('869f7783-15f6-4d39-b589-230936c9035a', None, None, None, None, None)] {{(pid=127601) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 13:57:49.707926 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-ba23c32e-df21-4c8d-9c32-d0c292194c77 req-6574923d-3faa-447c-af89-6cac1dcce883 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d7ae43e4-11c8-417c-be73-4f9fb5c42f2c", "tag": "48757fc0-370f-4fb9-a0d2-0688efed54e8"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:57:49.721813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-ba23c32e-df21-4c8d-9c32-d0c292194c77 req-6574923d-3faa-447c-af89-6cac1dcce883 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:49.722455 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-ba23c32e-df21-4c8d-9c32-d0c292194c77 req-6574923d-3faa-447c-af89-6cac1dcce883 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:49.723029 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-ba23c32e-df21-4c8d-9c32-d0c292194c77 req-6574923d-3faa-447c-af89-6cac1dcce883 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:49.744715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-ba23c32e-df21-4c8d-9c32-d0c292194c77 req-6574923d-3faa-447c-af89-6cac1dcce883 service nova] Creating event network-changed:48757fc0-370f-4fb9-a0d2-0688efed54e8 for instance d7ae43e4-11c8-417c-be73-4f9fb5c42f2c on hv-cinder-85646 Sep 08 13:57:49.755231 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-ba23c32e-df21-4c8d-9c32-d0c292194c77 req-6574923d-3faa-447c-af89-6cac1dcce883 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.050893 Sep 08 13:57:49.757571 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 229/489] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:57:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:57:49.767479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c24dcf1f-6e92-4db2-b551-48189c6ed4cb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:49.768093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c24dcf1f-6e92-4db2-b551-48189c6ed4cb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:49.775031 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c24dcf1f-6e92-4db2-b551-48189c6ed4cb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:49.775510 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c24dcf1f-6e92-4db2-b551-48189c6ed4cb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:49.775963 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c24dcf1f-6e92-4db2-b551-48189c6ed4cb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:49.783796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c24dcf1f-6e92-4db2-b551-48189c6ed4cb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:49.783796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c24dcf1f-6e92-4db2-b551-48189c6ed4cb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:49.783796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c24dcf1f-6e92-4db2-b551-48189c6ed4cb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:49.924432 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c24dcf1f-6e92-4db2-b551-48189c6ed4cb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:49.932540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c24dcf1f-6e92-4db2-b551-48189c6ed4cb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.173696 Sep 08 13:57:49.933103 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 230/490] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:49 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:50.159018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8321d8cc-9d6d-45bd-919f-a4f07a0be3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:50.159856 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8321d8cc-9d6d-45bd-919f-a4f07a0be3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:50.167670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8321d8cc-9d6d-45bd-919f-a4f07a0be3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:50.168260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8321d8cc-9d6d-45bd-919f-a4f07a0be3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:50.170915 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8321d8cc-9d6d-45bd-919f-a4f07a0be3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:50.172038 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8321d8cc-9d6d-45bd-919f-a4f07a0be3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:50.172792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8321d8cc-9d6d-45bd-919f-a4f07a0be3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:50.173466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8321d8cc-9d6d-45bd-919f-a4f07a0be3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:50.308940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8321d8cc-9d6d-45bd-919f-a4f07a0be3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:50.317939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8321d8cc-9d6d-45bd-919f-a4f07a0be3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:50.319464 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8321d8cc-9d6d-45bd-919f-a4f07a0be3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:50.320802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8321d8cc-9d6d-45bd-919f-a4f07a0be3fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.167576 Sep 08 13:57:50.322193 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 231/491] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:50 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:50.339009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-8a70ea4f-2275-4cdc-a55a-fedc28be725e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Flavor limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 13:57:50.340091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-8a70ea4f-2275-4cdc-a55a-fedc28be725e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Image limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 13:57:50.345434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-8a70ea4f-2275-4cdc-a55a-fedc28be725e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Flavor pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 13:57:50.345434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-8a70ea4f-2275-4cdc-a55a-fedc28be725e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Image pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 13:57:50.345434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-8a70ea4f-2275-4cdc-a55a-fedc28be725e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 13:57:50.351137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a70ea4f-2275-4cdc-a55a-fedc28be725e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:50.351712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a70ea4f-2275-4cdc-a55a-fedc28be725e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:50.352267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a70ea4f-2275-4cdc-a55a-fedc28be725e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:50.353702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a70ea4f-2275-4cdc-a55a-fedc28be725e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:50.354267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a70ea4f-2275-4cdc-a55a-fedc28be725e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:50.354768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a70ea4f-2275-4cdc-a55a-fedc28be725e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:50.373799 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-8a70ea4f-2275-4cdc-a55a-fedc28be725e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Getting quotas for project 5be47c3b2c4c4fc2a1db117433cbcfc4. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:57:50.375394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:50.379198 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-8a70ea4f-2275-4cdc-a55a-fedc28be725e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Getting quotas for user f2026170f4674b63a37534c14689bd10 and project 5be47c3b2c4c4fc2a1db117433cbcfc4. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:57:50.451433 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8a70ea4f-2275-4cdc-a55a-fedc28be725e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Going to run 1 instances... {{(pid=127601) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 13:57:50.453683 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8a70ea4f-2275-4cdc-a55a-fedc28be725e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='2012ff93-344d-4a13-879f-779050ac0a2d',volume_size=None,volume_type=None)] {{(pid=127601) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 13:57:50.937937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8a70ea4f-2275-4cdc-a55a-fedc28be725e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.480814 Sep 08 13:57:50.943357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 261/492] 10.222.0.25 () {60 vars in 1147 bytes} [Thu Sep 8 13:57:49 2022] POST /compute/v2.1/servers => generated 406 bytes in 1487 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 08 13:57:50.965368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4f377ed9-a7ad-4b51-abc0-2a755685c4d6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:50.965990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4f377ed9-a7ad-4b51-abc0-2a755685c4d6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:50.975232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f377ed9-a7ad-4b51-abc0-2a755685c4d6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:50.975732 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f377ed9-a7ad-4b51-abc0-2a755685c4d6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:50.976181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f377ed9-a7ad-4b51-abc0-2a755685c4d6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:50.976889 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f377ed9-a7ad-4b51-abc0-2a755685c4d6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:50.977382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f377ed9-a7ad-4b51-abc0-2a755685c4d6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:50.978476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f377ed9-a7ad-4b51-abc0-2a755685c4d6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:50.980208 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-be99fe00-5a99-4eec-931c-67574f9dd504 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:50.980857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-be99fe00-5a99-4eec-931c-67574f9dd504 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:51.224646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-be99fe00-5a99-4eec-931c-67574f9dd504 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:51.234450 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-be99fe00-5a99-4eec-931c-67574f9dd504 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1088 microversion: 2.1 time: 0.281740 Sep 08 13:57:51.235376 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 232/493] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:50 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1088 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:51.241391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-79afc524-95f6-4573-b656-6be08f7dbae0 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "fa5de96f-82e6-44dd-a208-403b6087dd70", "device": "/dev/sdb"}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:57:51.244852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-79afc524-95f6-4573-b656-6be08f7dbae0 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:51.254225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79afc524-95f6-4573-b656-6be08f7dbae0 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:51.255001 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79afc524-95f6-4573-b656-6be08f7dbae0 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:51.257300 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79afc524-95f6-4573-b656-6be08f7dbae0 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:51.257708 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79afc524-95f6-4573-b656-6be08f7dbae0 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:51.259000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79afc524-95f6-4573-b656-6be08f7dbae0 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:51.259656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79afc524-95f6-4573-b656-6be08f7dbae0 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:51.296687 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4f377ed9-a7ad-4b51-abc0-2a755685c4d6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:51.305252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4f377ed9-a7ad-4b51-abc0-2a755685c4d6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.353068 Sep 08 13:57:51.311589 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 262/494] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:50 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:51.346901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-853e0ea3-bdd1-4606-9028-6afb068d29a6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:51.347716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-853e0ea3-bdd1-4606-9028-6afb068d29a6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:51.356160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-853e0ea3-bdd1-4606-9028-6afb068d29a6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:51.357474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-853e0ea3-bdd1-4606-9028-6afb068d29a6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:51.358489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-853e0ea3-bdd1-4606-9028-6afb068d29a6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:51.361619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-853e0ea3-bdd1-4606-9028-6afb068d29a6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:51.362454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-853e0ea3-bdd1-4606-9028-6afb068d29a6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:51.363084 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-853e0ea3-bdd1-4606-9028-6afb068d29a6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:51.632162 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-853e0ea3-bdd1-4606-9028-6afb068d29a6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:51.658716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-853e0ea3-bdd1-4606-9028-6afb068d29a6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:51.661234 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-853e0ea3-bdd1-4606-9028-6afb068d29a6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:51.662612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-853e0ea3-bdd1-4606-9028-6afb068d29a6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.320144 Sep 08 13:57:51.663899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 263/495] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:51 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:52.257231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-24bbf9a7-1681-41fe-a910-42635b44cca5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:52.258646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-24bbf9a7-1681-41fe-a910-42635b44cca5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:52.605831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-24bbf9a7-1681-41fe-a910-42635b44cca5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:52.616939 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-24bbf9a7-1681-41fe-a910-42635b44cca5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1088 microversion: 2.1 time: 0.360251 Sep 08 13:57:52.616939 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 264/496] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:52 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1088 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:52.620535 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-38a9e8eb-1962-4ef2-a65f-6d12f26bf438 req-a7b5548e-7096-49cf-83b9-f2ab1134495c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6c3f043c-0662-4b63-bbd5-3a0f68c89b94", "tag": "0476c717-e075-4f94-b8e8-3fa5f88138b1"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:57:52.632496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-38a9e8eb-1962-4ef2-a65f-6d12f26bf438 req-a7b5548e-7096-49cf-83b9-f2ab1134495c service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:52.633024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-38a9e8eb-1962-4ef2-a65f-6d12f26bf438 req-a7b5548e-7096-49cf-83b9-f2ab1134495c service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:52.633505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-38a9e8eb-1962-4ef2-a65f-6d12f26bf438 req-a7b5548e-7096-49cf-83b9-f2ab1134495c service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:52.646615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-38a9e8eb-1962-4ef2-a65f-6d12f26bf438 req-a7b5548e-7096-49cf-83b9-f2ab1134495c service nova] Creating event network-changed:0476c717-e075-4f94-b8e8-3fa5f88138b1 for instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 on hv-cinder-85646 Sep 08 13:57:52.652963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-38a9e8eb-1962-4ef2-a65f-6d12f26bf438 req-a7b5548e-7096-49cf-83b9-f2ab1134495c service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036772 Sep 08 13:57:52.660224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 265/497] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:57:52 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:57:52.665420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a0d99ef4-2d61-4ebf-939c-820f8093c8da tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:52.666656 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a0d99ef4-2d61-4ebf-939c-820f8093c8da tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:52.673490 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a0d99ef4-2d61-4ebf-939c-820f8093c8da tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:52.674297 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a0d99ef4-2d61-4ebf-939c-820f8093c8da tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:52.674910 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a0d99ef4-2d61-4ebf-939c-820f8093c8da tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:52.675796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a0d99ef4-2d61-4ebf-939c-820f8093c8da tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:52.676634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a0d99ef4-2d61-4ebf-939c-820f8093c8da tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:52.677500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a0d99ef4-2d61-4ebf-939c-820f8093c8da tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:52.939026 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a0d99ef4-2d61-4ebf-939c-820f8093c8da tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:52.947759 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a0d99ef4-2d61-4ebf-939c-820f8093c8da tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.289241 Sep 08 13:57:52.952691 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 266/498] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:52 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:52.963805 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1d61cec2-c6e5-4887-9e47-d335c3bc2f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:52.964865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1d61cec2-c6e5-4887-9e47-d335c3bc2f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:52.971504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d61cec2-c6e5-4887-9e47-d335c3bc2f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:52.972256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d61cec2-c6e5-4887-9e47-d335c3bc2f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:52.972859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d61cec2-c6e5-4887-9e47-d335c3bc2f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:52.974201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d61cec2-c6e5-4887-9e47-d335c3bc2f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:52.974861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d61cec2-c6e5-4887-9e47-d335c3bc2f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:52.975499 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d61cec2-c6e5-4887-9e47-d335c3bc2f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:53.142004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1d61cec2-c6e5-4887-9e47-d335c3bc2f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:53.152246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1d61cec2-c6e5-4887-9e47-d335c3bc2f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:53.154270 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1d61cec2-c6e5-4887-9e47-d335c3bc2f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:53.158763 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1d61cec2-c6e5-4887-9e47-d335c3bc2f06 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.202857 Sep 08 13:57:53.158763 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 267/499] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:52 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:53.637813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1273ce3d-a89b-4018-adb5-3bd2ecae2585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:53.638620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1273ce3d-a89b-4018-adb5-3bd2ecae2585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:53.646061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1273ce3d-a89b-4018-adb5-3bd2ecae2585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:53.646632 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1273ce3d-a89b-4018-adb5-3bd2ecae2585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:53.647063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1273ce3d-a89b-4018-adb5-3bd2ecae2585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:53.647806 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1273ce3d-a89b-4018-adb5-3bd2ecae2585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:53.648256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1273ce3d-a89b-4018-adb5-3bd2ecae2585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:53.648674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1273ce3d-a89b-4018-adb5-3bd2ecae2585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:53.711900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1273ce3d-a89b-4018-adb5-3bd2ecae2585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:53.719977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1273ce3d-a89b-4018-adb5-3bd2ecae2585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.086695 Sep 08 13:57:53.721945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 268/500] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:53 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:53.973766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-878b396a-1bc3-4a0b-8b03-7824e0d430e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:53.974585 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-878b396a-1bc3-4a0b-8b03-7824e0d430e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:53.982429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-878b396a-1bc3-4a0b-8b03-7824e0d430e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:53.983209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-878b396a-1bc3-4a0b-8b03-7824e0d430e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:53.983813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-878b396a-1bc3-4a0b-8b03-7824e0d430e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:53.985446 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-878b396a-1bc3-4a0b-8b03-7824e0d430e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:53.986603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-878b396a-1bc3-4a0b-8b03-7824e0d430e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:53.987203 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-878b396a-1bc3-4a0b-8b03-7824e0d430e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:54.128179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-878b396a-1bc3-4a0b-8b03-7824e0d430e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:54.137828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-878b396a-1bc3-4a0b-8b03-7824e0d430e5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.170260 Sep 08 13:57:54.150536 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 269/501] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:53 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:54.184051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0b2c57d3-42da-4068-8aa1-d4b2a71239a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:54.184651 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0b2c57d3-42da-4068-8aa1-d4b2a71239a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:54.190617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b2c57d3-42da-4068-8aa1-d4b2a71239a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:54.197639 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b2c57d3-42da-4068-8aa1-d4b2a71239a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:54.197639 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b2c57d3-42da-4068-8aa1-d4b2a71239a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:54.197639 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b2c57d3-42da-4068-8aa1-d4b2a71239a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:54.197639 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b2c57d3-42da-4068-8aa1-d4b2a71239a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:54.198014 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b2c57d3-42da-4068-8aa1-d4b2a71239a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:54.365089 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0b2c57d3-42da-4068-8aa1-d4b2a71239a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:54.375357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0b2c57d3-42da-4068-8aa1-d4b2a71239a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:54.377087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0b2c57d3-42da-4068-8aa1-d4b2a71239a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:54.378594 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0b2c57d3-42da-4068-8aa1-d4b2a71239a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.198411 Sep 08 13:57:54.379362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 270/502] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:54 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:54.745386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-44eacda4-be77-4272-951f-2f51df5e202f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:54.747010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-44eacda4-be77-4272-951f-2f51df5e202f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:54.754783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44eacda4-be77-4272-951f-2f51df5e202f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:54.755602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44eacda4-be77-4272-951f-2f51df5e202f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:54.756271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44eacda4-be77-4272-951f-2f51df5e202f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:54.757204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44eacda4-be77-4272-951f-2f51df5e202f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:54.757921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44eacda4-be77-4272-951f-2f51df5e202f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:54.759381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44eacda4-be77-4272-951f-2f51df5e202f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:54.830127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-44eacda4-be77-4272-951f-2f51df5e202f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:54.838645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-44eacda4-be77-4272-951f-2f51df5e202f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.098001 Sep 08 13:57:54.839214 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 271/503] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:54 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:55.161065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-08f7c733-8b6a-4466-a85f-8a00e0e4398b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:55.161806 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-08f7c733-8b6a-4466-a85f-8a00e0e4398b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:55.168577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08f7c733-8b6a-4466-a85f-8a00e0e4398b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:55.169124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08f7c733-8b6a-4466-a85f-8a00e0e4398b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:55.169577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08f7c733-8b6a-4466-a85f-8a00e0e4398b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:55.170318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08f7c733-8b6a-4466-a85f-8a00e0e4398b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:55.170779 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08f7c733-8b6a-4466-a85f-8a00e0e4398b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:55.171233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08f7c733-8b6a-4466-a85f-8a00e0e4398b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:55.340120 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-08f7c733-8b6a-4466-a85f-8a00e0e4398b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:55.360599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-08f7c733-8b6a-4466-a85f-8a00e0e4398b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.203210 Sep 08 13:57:55.382354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 272/504] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:55 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:55.409010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-56dc5d89-91b0-46d9-8d3d-36287862a2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:55.409645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-56dc5d89-91b0-46d9-8d3d-36287862a2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:55.417362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56dc5d89-91b0-46d9-8d3d-36287862a2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:55.417855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56dc5d89-91b0-46d9-8d3d-36287862a2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:55.418401 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56dc5d89-91b0-46d9-8d3d-36287862a2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:55.419107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56dc5d89-91b0-46d9-8d3d-36287862a2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:55.419562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56dc5d89-91b0-46d9-8d3d-36287862a2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:55.420093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56dc5d89-91b0-46d9-8d3d-36287862a2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:55.616552 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-56dc5d89-91b0-46d9-8d3d-36287862a2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:55.627338 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-56dc5d89-91b0-46d9-8d3d-36287862a2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:55.629099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-56dc5d89-91b0-46d9-8d3d-36287862a2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:55.630507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-56dc5d89-91b0-46d9-8d3d-36287862a2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.227543 Sep 08 13:57:55.631228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 273/505] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:55 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:57:55.860981 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fb105426-bd7f-411e-a2ab-dedce37408b5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:55.863109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fb105426-bd7f-411e-a2ab-dedce37408b5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:55.870379 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb105426-bd7f-411e-a2ab-dedce37408b5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:55.870873 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb105426-bd7f-411e-a2ab-dedce37408b5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:55.871310 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb105426-bd7f-411e-a2ab-dedce37408b5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:55.872015 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb105426-bd7f-411e-a2ab-dedce37408b5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:55.872469 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb105426-bd7f-411e-a2ab-dedce37408b5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:55.872895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb105426-bd7f-411e-a2ab-dedce37408b5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:55.934619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fb105426-bd7f-411e-a2ab-dedce37408b5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:55.942214 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fb105426-bd7f-411e-a2ab-dedce37408b5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.085123 Sep 08 13:57:55.942778 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 274/506] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:55 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:56.388926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d7107188-ecc5-47c9-b41a-0257e66645c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:56.389582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d7107188-ecc5-47c9-b41a-0257e66645c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:56.397795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d7107188-ecc5-47c9-b41a-0257e66645c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:56.423988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d7107188-ecc5-47c9-b41a-0257e66645c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:56.423988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d7107188-ecc5-47c9-b41a-0257e66645c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:56.423988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d7107188-ecc5-47c9-b41a-0257e66645c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:56.423988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d7107188-ecc5-47c9-b41a-0257e66645c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:56.424460 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d7107188-ecc5-47c9-b41a-0257e66645c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:56.424460 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:56.561944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d7107188-ecc5-47c9-b41a-0257e66645c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:56.570172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d7107188-ecc5-47c9-b41a-0257e66645c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.185196 Sep 08 13:57:56.570764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 275/507] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:56 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:56.657052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ed04b69c-9d8d-4412-858d-ad68b504bb03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:56.657674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ed04b69c-9d8d-4412-858d-ad68b504bb03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:56.665717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed04b69c-9d8d-4412-858d-ad68b504bb03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:56.666293 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed04b69c-9d8d-4412-858d-ad68b504bb03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:56.666733 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed04b69c-9d8d-4412-858d-ad68b504bb03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:56.667465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed04b69c-9d8d-4412-858d-ad68b504bb03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:56.667948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed04b69c-9d8d-4412-858d-ad68b504bb03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:56.668379 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed04b69c-9d8d-4412-858d-ad68b504bb03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:56.814305 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ed04b69c-9d8d-4412-858d-ad68b504bb03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:56.823938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ed04b69c-9d8d-4412-858d-ad68b504bb03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:56.825381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ed04b69c-9d8d-4412-858d-ad68b504bb03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:56.826591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ed04b69c-9d8d-4412-858d-ad68b504bb03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.174578 Sep 08 13:57:56.827154 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 276/508] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:56 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:56.965313 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-650189af-2d37-4678-a0ad-e58dc3a1b7ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:56.965961 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-650189af-2d37-4678-a0ad-e58dc3a1b7ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:56.972320 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-650189af-2d37-4678-a0ad-e58dc3a1b7ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:56.972804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-650189af-2d37-4678-a0ad-e58dc3a1b7ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:56.973233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-650189af-2d37-4678-a0ad-e58dc3a1b7ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:56.973924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-650189af-2d37-4678-a0ad-e58dc3a1b7ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:56.974405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-650189af-2d37-4678-a0ad-e58dc3a1b7ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:56.974824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-650189af-2d37-4678-a0ad-e58dc3a1b7ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:57.035870 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-650189af-2d37-4678-a0ad-e58dc3a1b7ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:57.044037 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-650189af-2d37-4678-a0ad-e58dc3a1b7ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.083149 Sep 08 13:57:57.044587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 277/509] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:56 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:57.606047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2df5c600-50d7-429a-83b3-c594a69e4042 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:57.607327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2df5c600-50d7-429a-83b3-c594a69e4042 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:57.618586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2df5c600-50d7-429a-83b3-c594a69e4042 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:57.618586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2df5c600-50d7-429a-83b3-c594a69e4042 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:57.618586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2df5c600-50d7-429a-83b3-c594a69e4042 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:57.618586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2df5c600-50d7-429a-83b3-c594a69e4042 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:57.620739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2df5c600-50d7-429a-83b3-c594a69e4042 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:57.620739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2df5c600-50d7-429a-83b3-c594a69e4042 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:57.769256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2df5c600-50d7-429a-83b3-c594a69e4042 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:57.777741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2df5c600-50d7-429a-83b3-c594a69e4042 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.187977 Sep 08 13:57:57.778368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 278/510] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:57 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:57.851932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-47c03953-9bac-459c-a05e-e82378fc4eb7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:57.852558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-47c03953-9bac-459c-a05e-e82378fc4eb7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:57.859177 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47c03953-9bac-459c-a05e-e82378fc4eb7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:57.859636 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47c03953-9bac-459c-a05e-e82378fc4eb7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:57.860068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47c03953-9bac-459c-a05e-e82378fc4eb7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:57.860755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47c03953-9bac-459c-a05e-e82378fc4eb7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:57.861204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47c03953-9bac-459c-a05e-e82378fc4eb7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:57.861619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47c03953-9bac-459c-a05e-e82378fc4eb7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:58.002329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-47c03953-9bac-459c-a05e-e82378fc4eb7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:58.011007 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-47c03953-9bac-459c-a05e-e82378fc4eb7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:58.012463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-47c03953-9bac-459c-a05e-e82378fc4eb7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:58.013611 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-47c03953-9bac-459c-a05e-e82378fc4eb7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.165688 Sep 08 13:57:58.014257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 279/511] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:57 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:58.069652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ad788d8b-39e9-49cc-b99a-6f7ba6279f25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:58.070799 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ad788d8b-39e9-49cc-b99a-6f7ba6279f25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:58.078993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad788d8b-39e9-49cc-b99a-6f7ba6279f25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:58.079484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad788d8b-39e9-49cc-b99a-6f7ba6279f25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:58.079921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad788d8b-39e9-49cc-b99a-6f7ba6279f25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:58.080623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad788d8b-39e9-49cc-b99a-6f7ba6279f25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:58.081828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad788d8b-39e9-49cc-b99a-6f7ba6279f25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:58.081828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad788d8b-39e9-49cc-b99a-6f7ba6279f25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:58.151023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ad788d8b-39e9-49cc-b99a-6f7ba6279f25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:58.159201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ad788d8b-39e9-49cc-b99a-6f7ba6279f25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.095842 Sep 08 13:57:58.159771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 280/512] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:58 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:57:58.792347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-4b0e83c3-3a70-49c7-8a9b-522e79f175a3 req-0e63d99b-8ea1-4297-9466-3c908f3d1d29 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c14f6817-e3ab-4a36-ba23-590498bf1b25", "name": "network-vif-plugged", "status": "completed", "tag": "4894f431-b6f6-46b3-95db-e9eb88227e33"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:57:58.802614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-4b0e83c3-3a70-49c7-8a9b-522e79f175a3 req-0e63d99b-8ea1-4297-9466-3c908f3d1d29 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:58.803218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-4b0e83c3-3a70-49c7-8a9b-522e79f175a3 req-0e63d99b-8ea1-4297-9466-3c908f3d1d29 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:58.803721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-4b0e83c3-3a70-49c7-8a9b-522e79f175a3 req-0e63d99b-8ea1-4297-9466-3c908f3d1d29 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:58.815970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-4b0e83c3-3a70-49c7-8a9b-522e79f175a3 req-0e63d99b-8ea1-4297-9466-3c908f3d1d29 service nova] Creating event network-vif-plugged:4894f431-b6f6-46b3-95db-e9eb88227e33 for instance c14f6817-e3ab-4a36-ba23-590498bf1b25 on hv-cinder-85646 Sep 08 13:57:58.821309 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-4b0e83c3-3a70-49c7-8a9b-522e79f175a3 req-0e63d99b-8ea1-4297-9466-3c908f3d1d29 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032846 Sep 08 13:57:58.826468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 281/513] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:57:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:57:58.827917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-aca88a8f-f705-49ed-93b6-47ff1f272dad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:58.829204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-aca88a8f-f705-49ed-93b6-47ff1f272dad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:58.834933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aca88a8f-f705-49ed-93b6-47ff1f272dad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:58.835429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aca88a8f-f705-49ed-93b6-47ff1f272dad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:58.835870 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aca88a8f-f705-49ed-93b6-47ff1f272dad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:58.836575 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aca88a8f-f705-49ed-93b6-47ff1f272dad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:58.837037 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aca88a8f-f705-49ed-93b6-47ff1f272dad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:58.838082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aca88a8f-f705-49ed-93b6-47ff1f272dad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:59.000347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-aca88a8f-f705-49ed-93b6-47ff1f272dad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:59.007949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-aca88a8f-f705-49ed-93b6-47ff1f272dad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.184224 Sep 08 13:57:59.008549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 282/514] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:58 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:57:59.038524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-745ff920-583b-4e20-8dd0-a4c6fd2de029 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:59.039147 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-745ff920-583b-4e20-8dd0-a4c6fd2de029 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:59.044969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-745ff920-583b-4e20-8dd0-a4c6fd2de029 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:59.045453 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-745ff920-583b-4e20-8dd0-a4c6fd2de029 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:59.045889 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-745ff920-583b-4e20-8dd0-a4c6fd2de029 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:59.046609 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-745ff920-583b-4e20-8dd0-a4c6fd2de029 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:59.047252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-745ff920-583b-4e20-8dd0-a4c6fd2de029 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:59.047830 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-745ff920-583b-4e20-8dd0-a4c6fd2de029 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:59.189888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-745ff920-583b-4e20-8dd0-a4c6fd2de029 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:59.200526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-745ff920-583b-4e20-8dd0-a4c6fd2de029 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:59.203368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-745ff920-583b-4e20-8dd0-a4c6fd2de029 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:59.204829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-745ff920-583b-4e20-8dd0-a4c6fd2de029 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.170114 Sep 08 13:57:59.210300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 283/515] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:57:59 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:57:59.214312 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7757d85c-ca9b-4fc1-adcd-68062b430385 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:57:59.215196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7757d85c-ca9b-4fc1-adcd-68062b430385 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:57:59.226206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7757d85c-ca9b-4fc1-adcd-68062b430385 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:59.226713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7757d85c-ca9b-4fc1-adcd-68062b430385 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:59.227158 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7757d85c-ca9b-4fc1-adcd-68062b430385 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:59.227875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7757d85c-ca9b-4fc1-adcd-68062b430385 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:57:59.228340 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7757d85c-ca9b-4fc1-adcd-68062b430385 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:57:59.228772 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7757d85c-ca9b-4fc1-adcd-68062b430385 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:57:59.291407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7757d85c-ca9b-4fc1-adcd-68062b430385 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:57:59.299523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7757d85c-ca9b-4fc1-adcd-68062b430385 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.092340 Sep 08 13:57:59.300117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 284/516] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:57:59 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:00.031819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-072e1a4a-4f93-4f7b-a01a-c6b4a46d4509 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:00.032447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-072e1a4a-4f93-4f7b-a01a-c6b4a46d4509 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:00.038847 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-072e1a4a-4f93-4f7b-a01a-c6b4a46d4509 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:00.039334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-072e1a4a-4f93-4f7b-a01a-c6b4a46d4509 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:00.039771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-072e1a4a-4f93-4f7b-a01a-c6b4a46d4509 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:00.040461 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-072e1a4a-4f93-4f7b-a01a-c6b4a46d4509 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:00.041502 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-072e1a4a-4f93-4f7b-a01a-c6b4a46d4509 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:00.041925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-072e1a4a-4f93-4f7b-a01a-c6b4a46d4509 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:00.185179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-072e1a4a-4f93-4f7b-a01a-c6b4a46d4509 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:00.194682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-072e1a4a-4f93-4f7b-a01a-c6b4a46d4509 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.166833 Sep 08 13:58:00.205123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 285/517] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:00 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:00.230739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-791008ec-4ec8-4da7-afcf-1016067302f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:00.231345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-791008ec-4ec8-4da7-afcf-1016067302f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:00.237241 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-791008ec-4ec8-4da7-afcf-1016067302f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:00.237720 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-791008ec-4ec8-4da7-afcf-1016067302f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:00.238180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-791008ec-4ec8-4da7-afcf-1016067302f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:00.238866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-791008ec-4ec8-4da7-afcf-1016067302f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:00.239320 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-791008ec-4ec8-4da7-afcf-1016067302f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:00.239741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-791008ec-4ec8-4da7-afcf-1016067302f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:00.363421 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=2 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:00.376500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-791008ec-4ec8-4da7-afcf-1016067302f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:00.385884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-791008ec-4ec8-4da7-afcf-1016067302f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:00.387411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-791008ec-4ec8-4da7-afcf-1016067302f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:00.388531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-791008ec-4ec8-4da7-afcf-1016067302f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.161779 Sep 08 13:58:00.389092 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 286/518] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:00 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:00.393424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1b6e3bf5-a408-4edf-abeb-acdfa8f43f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:00.394024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1b6e3bf5-a408-4edf-abeb-acdfa8f43f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:00.400456 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1b6e3bf5-a408-4edf-abeb-acdfa8f43f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:00.401780 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1b6e3bf5-a408-4edf-abeb-acdfa8f43f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:00.401780 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1b6e3bf5-a408-4edf-abeb-acdfa8f43f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:00.402061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1b6e3bf5-a408-4edf-abeb-acdfa8f43f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:00.402542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1b6e3bf5-a408-4edf-abeb-acdfa8f43f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:00.402972 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1b6e3bf5-a408-4edf-abeb-acdfa8f43f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:00.474613 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1b6e3bf5-a408-4edf-abeb-acdfa8f43f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:00.498944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1b6e3bf5-a408-4edf-abeb-acdfa8f43f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.092674 Sep 08 13:58:00.498944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 287/519] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:00 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:01.217276 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3a27a992-7e82-401b-adbc-01a0745da893 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:01.217899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3a27a992-7e82-401b-adbc-01a0745da893 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:01.225045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3a27a992-7e82-401b-adbc-01a0745da893 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:01.225535 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3a27a992-7e82-401b-adbc-01a0745da893 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:01.225970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3a27a992-7e82-401b-adbc-01a0745da893 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:01.226703 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3a27a992-7e82-401b-adbc-01a0745da893 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:01.227152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3a27a992-7e82-401b-adbc-01a0745da893 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:01.227577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3a27a992-7e82-401b-adbc-01a0745da893 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:01.230553 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=2 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:01.253088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=24 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:01.362781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3a27a992-7e82-401b-adbc-01a0745da893 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:01.371358 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3a27a992-7e82-401b-adbc-01a0745da893 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.157737 Sep 08 13:58:01.371952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 288/520] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:01 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:01.414686 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7aabc618-fcac-417e-97ae-5ef3828d324f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:01.415315 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7aabc618-fcac-417e-97ae-5ef3828d324f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:01.421827 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7aabc618-fcac-417e-97ae-5ef3828d324f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:01.422408 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7aabc618-fcac-417e-97ae-5ef3828d324f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:01.422848 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7aabc618-fcac-417e-97ae-5ef3828d324f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:01.423534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7aabc618-fcac-417e-97ae-5ef3828d324f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:01.423988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7aabc618-fcac-417e-97ae-5ef3828d324f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:01.424413 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7aabc618-fcac-417e-97ae-5ef3828d324f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:01.438570 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:01.557937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7aabc618-fcac-417e-97ae-5ef3828d324f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:01.567404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7aabc618-fcac-417e-97ae-5ef3828d324f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:01.569790 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7aabc618-fcac-417e-97ae-5ef3828d324f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:01.569996 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7aabc618-fcac-417e-97ae-5ef3828d324f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.159288 Sep 08 13:58:01.582169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 289/521] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:01 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:01.582169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-86fc9994-451f-402f-ab58-d205cde2f4ec tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:01.582169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-86fc9994-451f-402f-ab58-d205cde2f4ec tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:01.582169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-86fc9994-451f-402f-ab58-d205cde2f4ec tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:01.582664 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-86fc9994-451f-402f-ab58-d205cde2f4ec tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:01.583035 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-86fc9994-451f-402f-ab58-d205cde2f4ec tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:01.585591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-86fc9994-451f-402f-ab58-d205cde2f4ec tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:01.585591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-86fc9994-451f-402f-ab58-d205cde2f4ec tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:01.585591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-86fc9994-451f-402f-ab58-d205cde2f4ec tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:01.647210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-86fc9994-451f-402f-ab58-d205cde2f4ec tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:01.660763 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-86fc9994-451f-402f-ab58-d205cde2f4ec tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.088771 Sep 08 13:58:01.661350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 290/522] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:01 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:02.393909 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b6ea2e47-283f-4d39-85ab-62d1e50efadc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:02.394603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b6ea2e47-283f-4d39-85ab-62d1e50efadc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:02.401058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b6ea2e47-283f-4d39-85ab-62d1e50efadc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:02.401551 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b6ea2e47-283f-4d39-85ab-62d1e50efadc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:02.401992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b6ea2e47-283f-4d39-85ab-62d1e50efadc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:02.402713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b6ea2e47-283f-4d39-85ab-62d1e50efadc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:02.403175 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b6ea2e47-283f-4d39-85ab-62d1e50efadc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:02.403600 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b6ea2e47-283f-4d39-85ab-62d1e50efadc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:02.541456 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b6ea2e47-283f-4d39-85ab-62d1e50efadc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:02.549334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b6ea2e47-283f-4d39-85ab-62d1e50efadc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.159473 Sep 08 13:58:02.549898 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 291/523] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:02 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:02.598589 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c1005523-fd45-4bfc-adb4-3de1393a27f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:02.599272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c1005523-fd45-4bfc-adb4-3de1393a27f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:02.605605 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1005523-fd45-4bfc-adb4-3de1393a27f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:02.606093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1005523-fd45-4bfc-adb4-3de1393a27f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:02.606557 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1005523-fd45-4bfc-adb4-3de1393a27f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:02.607344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1005523-fd45-4bfc-adb4-3de1393a27f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:02.607804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1005523-fd45-4bfc-adb4-3de1393a27f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:02.608250 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1005523-fd45-4bfc-adb4-3de1393a27f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:02.612388 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=8 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:02.623964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:02.744663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c1005523-fd45-4bfc-adb4-3de1393a27f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:02.753314 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c1005523-fd45-4bfc-adb4-3de1393a27f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:02.754807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c1005523-fd45-4bfc-adb4-3de1393a27f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:02.756041 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c1005523-fd45-4bfc-adb4-3de1393a27f3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.161610 Sep 08 13:58:02.756674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 292/524] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:02 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:02.761354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-47a6f629-57c8-488e-8050-24feb5131813 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:02.761972 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-47a6f629-57c8-488e-8050-24feb5131813 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:02.768006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47a6f629-57c8-488e-8050-24feb5131813 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:02.768543 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47a6f629-57c8-488e-8050-24feb5131813 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:02.768982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47a6f629-57c8-488e-8050-24feb5131813 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:02.769672 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47a6f629-57c8-488e-8050-24feb5131813 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:02.770151 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47a6f629-57c8-488e-8050-24feb5131813 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:02.770587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47a6f629-57c8-488e-8050-24feb5131813 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:02.833711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-47a6f629-57c8-488e-8050-24feb5131813 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:02.842549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-47a6f629-57c8-488e-8050-24feb5131813 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.084896 Sep 08 13:58:02.843104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 293/525] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:02 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:03.573396 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-22cbc924-d2c4-479a-9921-b7f9d75bcabc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:03.574013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-22cbc924-d2c4-479a-9921-b7f9d75bcabc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:03.580576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22cbc924-d2c4-479a-9921-b7f9d75bcabc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:03.581053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22cbc924-d2c4-479a-9921-b7f9d75bcabc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:03.581482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22cbc924-d2c4-479a-9921-b7f9d75bcabc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:03.582184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22cbc924-d2c4-479a-9921-b7f9d75bcabc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:03.582667 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22cbc924-d2c4-479a-9921-b7f9d75bcabc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:03.583085 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22cbc924-d2c4-479a-9921-b7f9d75bcabc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:03.722691 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-22cbc924-d2c4-479a-9921-b7f9d75bcabc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:03.730498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-22cbc924-d2c4-479a-9921-b7f9d75bcabc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.161538 Sep 08 13:58:03.731069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 294/526] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:03 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:03.781349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f9c09ff4-384f-4672-a253-baf8aa9185c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:03.781956 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f9c09ff4-384f-4672-a253-baf8aa9185c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:03.788397 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9c09ff4-384f-4672-a253-baf8aa9185c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:03.788821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9c09ff4-384f-4672-a253-baf8aa9185c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:03.789248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9c09ff4-384f-4672-a253-baf8aa9185c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:03.789943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9c09ff4-384f-4672-a253-baf8aa9185c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:03.796432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9c09ff4-384f-4672-a253-baf8aa9185c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:03.796432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9c09ff4-384f-4672-a253-baf8aa9185c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:03.926781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f9c09ff4-384f-4672-a253-baf8aa9185c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:03.936101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f9c09ff4-384f-4672-a253-baf8aa9185c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:03.937547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f9c09ff4-384f-4672-a253-baf8aa9185c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:03.938752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f9c09ff4-384f-4672-a253-baf8aa9185c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.161608 Sep 08 13:58:03.939316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 295/527] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:03 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:03.944026 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d46c5173-a8f6-4a33-a721-339f3ed3316b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:03.944626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d46c5173-a8f6-4a33-a721-339f3ed3316b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:03.950437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d46c5173-a8f6-4a33-a721-339f3ed3316b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:03.950933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d46c5173-a8f6-4a33-a721-339f3ed3316b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:03.951371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d46c5173-a8f6-4a33-a721-339f3ed3316b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:03.952066 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d46c5173-a8f6-4a33-a721-339f3ed3316b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:03.952525 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d46c5173-a8f6-4a33-a721-339f3ed3316b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:03.952953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d46c5173-a8f6-4a33-a721-339f3ed3316b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:04.023057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d46c5173-a8f6-4a33-a721-339f3ed3316b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:04.031419 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d46c5173-a8f6-4a33-a721-339f3ed3316b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.091081 Sep 08 13:58:04.032010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 296/528] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:03 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:04.754018 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2d632512-31bb-4a15-960e-ae2cb8db50bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:04.754734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2d632512-31bb-4a15-960e-ae2cb8db50bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:04.761509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d632512-31bb-4a15-960e-ae2cb8db50bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:04.761991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d632512-31bb-4a15-960e-ae2cb8db50bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:04.762444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d632512-31bb-4a15-960e-ae2cb8db50bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:04.763127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d632512-31bb-4a15-960e-ae2cb8db50bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:04.763576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d632512-31bb-4a15-960e-ae2cb8db50bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:04.764005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d632512-31bb-4a15-960e-ae2cb8db50bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:04.897183 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2d632512-31bb-4a15-960e-ae2cb8db50bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:04.906170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2d632512-31bb-4a15-960e-ae2cb8db50bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.156003 Sep 08 13:58:04.907107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 297/529] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:04 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:04.965614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bd25323d-05b3-4413-bd5d-339278fc1444 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:04.966275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bd25323d-05b3-4413-bd5d-339278fc1444 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:04.972604 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd25323d-05b3-4413-bd5d-339278fc1444 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:04.974047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd25323d-05b3-4413-bd5d-339278fc1444 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:04.974520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd25323d-05b3-4413-bd5d-339278fc1444 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:04.975235 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd25323d-05b3-4413-bd5d-339278fc1444 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:04.975698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd25323d-05b3-4413-bd5d-339278fc1444 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:04.976243 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd25323d-05b3-4413-bd5d-339278fc1444 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:05.120833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bd25323d-05b3-4413-bd5d-339278fc1444 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:05.129033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bd25323d-05b3-4413-bd5d-339278fc1444 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:05.130490 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bd25323d-05b3-4413-bd5d-339278fc1444 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:05.131643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bd25323d-05b3-4413-bd5d-339278fc1444 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.170020 Sep 08 13:58:05.132205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 298/530] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:04 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:05.137377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-380a46f3-811f-4dcf-a69d-a4e0c626e965 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:05.138072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-380a46f3-811f-4dcf-a69d-a4e0c626e965 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:05.143861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-380a46f3-811f-4dcf-a69d-a4e0c626e965 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:05.144287 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-380a46f3-811f-4dcf-a69d-a4e0c626e965 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:05.144812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-380a46f3-811f-4dcf-a69d-a4e0c626e965 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:05.145458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-380a46f3-811f-4dcf-a69d-a4e0c626e965 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:05.145912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-380a46f3-811f-4dcf-a69d-a4e0c626e965 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:05.146390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-380a46f3-811f-4dcf-a69d-a4e0c626e965 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:05.213100 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-380a46f3-811f-4dcf-a69d-a4e0c626e965 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:05.220967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-380a46f3-811f-4dcf-a69d-a4e0c626e965 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.087744 Sep 08 13:58:05.221618 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 299/531] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:05 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:05.928793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cd78f446-0bc2-4400-a090-1c206061ced7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:05.929435 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cd78f446-0bc2-4400-a090-1c206061ced7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:05.935816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cd78f446-0bc2-4400-a090-1c206061ced7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:05.936319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cd78f446-0bc2-4400-a090-1c206061ced7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:05.936747 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cd78f446-0bc2-4400-a090-1c206061ced7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:05.937438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cd78f446-0bc2-4400-a090-1c206061ced7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:05.937898 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cd78f446-0bc2-4400-a090-1c206061ced7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:05.938362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cd78f446-0bc2-4400-a090-1c206061ced7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:06.088911 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cd78f446-0bc2-4400-a090-1c206061ced7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:06.098155 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cd78f446-0bc2-4400-a090-1c206061ced7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.172840 Sep 08 13:58:06.098764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 300/532] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:05 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:06.158328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ba818ad2-1c5b-45db-9377-6b60fbe8ba41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:06.158932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ba818ad2-1c5b-45db-9377-6b60fbe8ba41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:06.164950 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba818ad2-1c5b-45db-9377-6b60fbe8ba41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:06.165418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba818ad2-1c5b-45db-9377-6b60fbe8ba41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:06.165882 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba818ad2-1c5b-45db-9377-6b60fbe8ba41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:06.166630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba818ad2-1c5b-45db-9377-6b60fbe8ba41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:06.167078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba818ad2-1c5b-45db-9377-6b60fbe8ba41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:06.167493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba818ad2-1c5b-45db-9377-6b60fbe8ba41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:06.298688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ba818ad2-1c5b-45db-9377-6b60fbe8ba41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:06.307408 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ba818ad2-1c5b-45db-9377-6b60fbe8ba41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:06.308843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ba818ad2-1c5b-45db-9377-6b60fbe8ba41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:06.309997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ba818ad2-1c5b-45db-9377-6b60fbe8ba41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.156644 Sep 08 13:58:06.310587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 301/533] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:06 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:06.315024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9e719109-7fe9-4eff-b9f9-0e8fc87e24aa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:06.315621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9e719109-7fe9-4eff-b9f9-0e8fc87e24aa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:06.321104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e719109-7fe9-4eff-b9f9-0e8fc87e24aa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:06.324208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e719109-7fe9-4eff-b9f9-0e8fc87e24aa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:06.324208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e719109-7fe9-4eff-b9f9-0e8fc87e24aa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:06.324208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e719109-7fe9-4eff-b9f9-0e8fc87e24aa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:06.324208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e719109-7fe9-4eff-b9f9-0e8fc87e24aa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:06.324747 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e719109-7fe9-4eff-b9f9-0e8fc87e24aa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:06.390406 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9e719109-7fe9-4eff-b9f9-0e8fc87e24aa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:06.399312 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9e719109-7fe9-4eff-b9f9-0e8fc87e24aa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.087675 Sep 08 13:58:06.399896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 302/534] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:06 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:07.121144 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f3f84f56-6db5-46cd-b2d4-f39e9d47f807 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:07.121800 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f3f84f56-6db5-46cd-b2d4-f39e9d47f807 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:07.129405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f3f84f56-6db5-46cd-b2d4-f39e9d47f807 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:07.130475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f3f84f56-6db5-46cd-b2d4-f39e9d47f807 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:07.130681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f3f84f56-6db5-46cd-b2d4-f39e9d47f807 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:07.131869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f3f84f56-6db5-46cd-b2d4-f39e9d47f807 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:07.131869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f3f84f56-6db5-46cd-b2d4-f39e9d47f807 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:07.132234 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f3f84f56-6db5-46cd-b2d4-f39e9d47f807 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:07.276724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f3f84f56-6db5-46cd-b2d4-f39e9d47f807 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:07.284571 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f3f84f56-6db5-46cd-b2d4-f39e9d47f807 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.167875 Sep 08 13:58:07.285208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 303/535] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:07 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:07.334544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cb8d18e0-5fe3-4083-9c33-3d76de3caabc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:07.335155 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cb8d18e0-5fe3-4083-9c33-3d76de3caabc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:07.341682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cb8d18e0-5fe3-4083-9c33-3d76de3caabc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:07.342192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cb8d18e0-5fe3-4083-9c33-3d76de3caabc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:07.342644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cb8d18e0-5fe3-4083-9c33-3d76de3caabc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:07.343335 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cb8d18e0-5fe3-4083-9c33-3d76de3caabc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:07.343796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cb8d18e0-5fe3-4083-9c33-3d76de3caabc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:07.344238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cb8d18e0-5fe3-4083-9c33-3d76de3caabc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:07.490376 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cb8d18e0-5fe3-4083-9c33-3d76de3caabc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:07.498976 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cb8d18e0-5fe3-4083-9c33-3d76de3caabc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:07.500398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cb8d18e0-5fe3-4083-9c33-3d76de3caabc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:07.501548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cb8d18e0-5fe3-4083-9c33-3d76de3caabc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.171016 Sep 08 13:58:07.502100 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 304/536] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:07 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:58:07.506477 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-56d9ea1f-a911-4ec0-bfc9-17c833d380ae tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:07.507054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-56d9ea1f-a911-4ec0-bfc9-17c833d380ae tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:07.512489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56d9ea1f-a911-4ec0-bfc9-17c833d380ae tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:07.512981 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56d9ea1f-a911-4ec0-bfc9-17c833d380ae tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:07.513397 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56d9ea1f-a911-4ec0-bfc9-17c833d380ae tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:07.514074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56d9ea1f-a911-4ec0-bfc9-17c833d380ae tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:07.514561 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56d9ea1f-a911-4ec0-bfc9-17c833d380ae tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:07.514986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56d9ea1f-a911-4ec0-bfc9-17c833d380ae tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:07.577605 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-56d9ea1f-a911-4ec0-bfc9-17c833d380ae tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:07.585538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-56d9ea1f-a911-4ec0-bfc9-17c833d380ae tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.082472 Sep 08 13:58:07.586106 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 305/537] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:07 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:08.308854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-02a970b7-7c9c-463d-a04a-5972d72e6225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:08.310552 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-02a970b7-7c9c-463d-a04a-5972d72e6225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:08.311635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:08.316482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02a970b7-7c9c-463d-a04a-5972d72e6225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:08.316975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02a970b7-7c9c-463d-a04a-5972d72e6225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:08.317422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02a970b7-7c9c-463d-a04a-5972d72e6225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:08.318179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02a970b7-7c9c-463d-a04a-5972d72e6225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:08.318643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02a970b7-7c9c-463d-a04a-5972d72e6225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:08.319072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02a970b7-7c9c-463d-a04a-5972d72e6225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:08.453225 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-02a970b7-7c9c-463d-a04a-5972d72e6225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:08.461325 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-02a970b7-7c9c-463d-a04a-5972d72e6225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.157318 Sep 08 13:58:08.461916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 306/538] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:08 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:08.526535 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e87845d2-106b-40d5-845e-99b6ca984851 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:08.527050 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e87845d2-106b-40d5-845e-99b6ca984851 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:08.533282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e87845d2-106b-40d5-845e-99b6ca984851 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:08.533770 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e87845d2-106b-40d5-845e-99b6ca984851 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:08.534238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e87845d2-106b-40d5-845e-99b6ca984851 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:08.534927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e87845d2-106b-40d5-845e-99b6ca984851 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:08.535383 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e87845d2-106b-40d5-845e-99b6ca984851 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:08.535811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e87845d2-106b-40d5-845e-99b6ca984851 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:08.670489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e87845d2-106b-40d5-845e-99b6ca984851 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:08.679042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e87845d2-106b-40d5-845e-99b6ca984851 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:08.680496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e87845d2-106b-40d5-845e-99b6ca984851 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:08.681636 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e87845d2-106b-40d5-845e-99b6ca984851 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.159169 Sep 08 13:58:08.694616 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 307/539] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:08 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:08.694616 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a98bd324-ac3d-4b09-a735-0468cc9785e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:08.694616 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a98bd324-ac3d-4b09-a735-0468cc9785e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:08.694616 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a98bd324-ac3d-4b09-a735-0468cc9785e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:08.694616 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a98bd324-ac3d-4b09-a735-0468cc9785e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:08.695078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a98bd324-ac3d-4b09-a735-0468cc9785e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:08.695078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a98bd324-ac3d-4b09-a735-0468cc9785e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:08.695411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a98bd324-ac3d-4b09-a735-0468cc9785e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:08.695833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a98bd324-ac3d-4b09-a735-0468cc9785e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:08.756989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a98bd324-ac3d-4b09-a735-0468cc9785e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:08.765206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a98bd324-ac3d-4b09-a735-0468cc9785e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.081267 Sep 08 13:58:08.765780 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 308/540] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:08 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:09.483637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fb280754-220c-4480-8c0d-4e6fa9f8293b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:09.484335 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fb280754-220c-4480-8c0d-4e6fa9f8293b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:09.490702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb280754-220c-4480-8c0d-4e6fa9f8293b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:09.491186 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb280754-220c-4480-8c0d-4e6fa9f8293b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:09.491625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb280754-220c-4480-8c0d-4e6fa9f8293b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:09.492314 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb280754-220c-4480-8c0d-4e6fa9f8293b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:09.492774 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb280754-220c-4480-8c0d-4e6fa9f8293b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:09.493284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb280754-220c-4480-8c0d-4e6fa9f8293b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:09.496450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:09.647395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fb280754-220c-4480-8c0d-4e6fa9f8293b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:09.655355 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fb280754-220c-4480-8c0d-4e6fa9f8293b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.175659 Sep 08 13:58:09.655923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 309/541] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:09 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:09.708767 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-086826c9-e6dc-4e75-a561-620be4c582e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:09.709368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-086826c9-e6dc-4e75-a561-620be4c582e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:09.715264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-086826c9-e6dc-4e75-a561-620be4c582e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:09.715755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-086826c9-e6dc-4e75-a561-620be4c582e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:09.716196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-086826c9-e6dc-4e75-a561-620be4c582e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:09.716896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-086826c9-e6dc-4e75-a561-620be4c582e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:09.717351 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-086826c9-e6dc-4e75-a561-620be4c582e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:09.717778 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-086826c9-e6dc-4e75-a561-620be4c582e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:09.850355 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-086826c9-e6dc-4e75-a561-620be4c582e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:09.860350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-086826c9-e6dc-4e75-a561-620be4c582e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:09.861802 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-086826c9-e6dc-4e75-a561-620be4c582e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:09.863013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-086826c9-e6dc-4e75-a561-620be4c582e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.158058 Sep 08 13:58:09.863858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 310/542] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:09 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:09.868823 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-436f5139-15a9-49d2-ab0a-4f540553ea1a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:09.868823 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-436f5139-15a9-49d2-ab0a-4f540553ea1a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:09.875802 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-436f5139-15a9-49d2-ab0a-4f540553ea1a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:09.876364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-436f5139-15a9-49d2-ab0a-4f540553ea1a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:09.876805 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-436f5139-15a9-49d2-ab0a-4f540553ea1a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:09.877506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-436f5139-15a9-49d2-ab0a-4f540553ea1a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:09.877962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-436f5139-15a9-49d2-ab0a-4f540553ea1a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:09.878415 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-436f5139-15a9-49d2-ab0a-4f540553ea1a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:09.942397 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-436f5139-15a9-49d2-ab0a-4f540553ea1a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:09.951040 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-436f5139-15a9-49d2-ab0a-4f540553ea1a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.086345 Sep 08 13:58:09.951592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 311/543] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:09 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:10.677750 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0ecf8e26-3c19-4dc4-9ef3-f8627a8c6db5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:10.678400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0ecf8e26-3c19-4dc4-9ef3-f8627a8c6db5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:10.685245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ecf8e26-3c19-4dc4-9ef3-f8627a8c6db5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:10.685726 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ecf8e26-3c19-4dc4-9ef3-f8627a8c6db5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:10.686182 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ecf8e26-3c19-4dc4-9ef3-f8627a8c6db5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:10.686868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ecf8e26-3c19-4dc4-9ef3-f8627a8c6db5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:10.687321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ecf8e26-3c19-4dc4-9ef3-f8627a8c6db5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:10.687752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ecf8e26-3c19-4dc4-9ef3-f8627a8c6db5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:10.816734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0ecf8e26-3c19-4dc4-9ef3-f8627a8c6db5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:10.824876 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0ecf8e26-3c19-4dc4-9ef3-f8627a8c6db5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.151113 Sep 08 13:58:10.825430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 312/544] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:10 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:10.894832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1ddb439d-89aa-49e7-aced-cd6201e792c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:10.895353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1ddb439d-89aa-49e7-aced-cd6201e792c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:10.901845 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1ddb439d-89aa-49e7-aced-cd6201e792c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:10.902371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1ddb439d-89aa-49e7-aced-cd6201e792c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:10.902807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1ddb439d-89aa-49e7-aced-cd6201e792c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:10.903492 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1ddb439d-89aa-49e7-aced-cd6201e792c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:10.903944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1ddb439d-89aa-49e7-aced-cd6201e792c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:10.904366 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1ddb439d-89aa-49e7-aced-cd6201e792c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:11.033155 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1ddb439d-89aa-49e7-aced-cd6201e792c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:11.041724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1ddb439d-89aa-49e7-aced-cd6201e792c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:11.043274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1ddb439d-89aa-49e7-aced-cd6201e792c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:11.044425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1ddb439d-89aa-49e7-aced-cd6201e792c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.153519 Sep 08 13:58:11.045487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 313/545] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:10 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:11.050402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b7cee03e-2ff9-42b5-8826-8e79533e24cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:11.051008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b7cee03e-2ff9-42b5-8826-8e79533e24cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:11.056837 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7cee03e-2ff9-42b5-8826-8e79533e24cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:11.057841 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7cee03e-2ff9-42b5-8826-8e79533e24cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:11.058307 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7cee03e-2ff9-42b5-8826-8e79533e24cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:11.059005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7cee03e-2ff9-42b5-8826-8e79533e24cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:11.059524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7cee03e-2ff9-42b5-8826-8e79533e24cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:11.059988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7cee03e-2ff9-42b5-8826-8e79533e24cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:11.124204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b7cee03e-2ff9-42b5-8826-8e79533e24cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:11.131795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b7cee03e-2ff9-42b5-8826-8e79533e24cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.085362 Sep 08 13:58:11.132364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 314/546] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:11 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:11.848429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e316c64a-9207-4baa-97f6-c7a02c8816b3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:11.849426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e316c64a-9207-4baa-97f6-c7a02c8816b3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:11.855602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e316c64a-9207-4baa-97f6-c7a02c8816b3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:11.856093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e316c64a-9207-4baa-97f6-c7a02c8816b3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:11.856534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e316c64a-9207-4baa-97f6-c7a02c8816b3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:11.857230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e316c64a-9207-4baa-97f6-c7a02c8816b3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:11.857687 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e316c64a-9207-4baa-97f6-c7a02c8816b3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:11.858125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e316c64a-9207-4baa-97f6-c7a02c8816b3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:11.991042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e316c64a-9207-4baa-97f6-c7a02c8816b3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:11.998630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e316c64a-9207-4baa-97f6-c7a02c8816b3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.154833 Sep 08 13:58:11.999186 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 315/547] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:11 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:12.070647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-02446112-cbe6-4ad8-a241-a149a705c05c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:12.071244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-02446112-cbe6-4ad8-a241-a149a705c05c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:12.077629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02446112-cbe6-4ad8-a241-a149a705c05c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:12.078106 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02446112-cbe6-4ad8-a241-a149a705c05c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:12.078564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02446112-cbe6-4ad8-a241-a149a705c05c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:12.079246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02446112-cbe6-4ad8-a241-a149a705c05c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:12.079712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02446112-cbe6-4ad8-a241-a149a705c05c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:12.080338 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02446112-cbe6-4ad8-a241-a149a705c05c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:12.208528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-02446112-cbe6-4ad8-a241-a149a705c05c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:12.217193 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-02446112-cbe6-4ad8-a241-a149a705c05c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:12.218650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-02446112-cbe6-4ad8-a241-a149a705c05c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:12.219904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-02446112-cbe6-4ad8-a241-a149a705c05c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.153130 Sep 08 13:58:12.220462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 316/548] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:12 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:12.230192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5153de84-5e65-4ed6-9c5d-0f8374c2a3fa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:12.230192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5153de84-5e65-4ed6-9c5d-0f8374c2a3fa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:12.232631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5153de84-5e65-4ed6-9c5d-0f8374c2a3fa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:12.233119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5153de84-5e65-4ed6-9c5d-0f8374c2a3fa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:12.233553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5153de84-5e65-4ed6-9c5d-0f8374c2a3fa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:12.234303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5153de84-5e65-4ed6-9c5d-0f8374c2a3fa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:12.234723 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5153de84-5e65-4ed6-9c5d-0f8374c2a3fa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:12.235431 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5153de84-5e65-4ed6-9c5d-0f8374c2a3fa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:12.300403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5153de84-5e65-4ed6-9c5d-0f8374c2a3fa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:12.308697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5153de84-5e65-4ed6-9c5d-0f8374c2a3fa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.087244 Sep 08 13:58:12.309279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 317/549] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:12 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:13.020928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-24addcb5-a1ac-439b-8d7c-e2c2419308b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:13.021575 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-24addcb5-a1ac-439b-8d7c-e2c2419308b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:13.028707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-24addcb5-a1ac-439b-8d7c-e2c2419308b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:13.029193 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-24addcb5-a1ac-439b-8d7c-e2c2419308b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:13.029628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-24addcb5-a1ac-439b-8d7c-e2c2419308b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:13.030603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-24addcb5-a1ac-439b-8d7c-e2c2419308b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:13.031059 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-24addcb5-a1ac-439b-8d7c-e2c2419308b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:13.031485 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-24addcb5-a1ac-439b-8d7c-e2c2419308b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:13.159123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-24addcb5-a1ac-439b-8d7c-e2c2419308b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:13.166781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-24addcb5-a1ac-439b-8d7c-e2c2419308b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.149752 Sep 08 13:58:13.167348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 318/550] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:13 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:13.245603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ef875710-2a95-4959-a89f-7071f9fd3557 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:13.246255 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ef875710-2a95-4959-a89f-7071f9fd3557 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:13.252054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef875710-2a95-4959-a89f-7071f9fd3557 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:13.252535 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef875710-2a95-4959-a89f-7071f9fd3557 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:13.252968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef875710-2a95-4959-a89f-7071f9fd3557 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:13.253651 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef875710-2a95-4959-a89f-7071f9fd3557 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:13.254104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef875710-2a95-4959-a89f-7071f9fd3557 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:13.254552 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef875710-2a95-4959-a89f-7071f9fd3557 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:13.384498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ef875710-2a95-4959-a89f-7071f9fd3557 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:13.393747 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ef875710-2a95-4959-a89f-7071f9fd3557 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:13.395224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ef875710-2a95-4959-a89f-7071f9fd3557 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:13.396387 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ef875710-2a95-4959-a89f-7071f9fd3557 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.155150 Sep 08 13:58:13.396949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 319/551] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:13 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:13.401893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5831ee19-2ef6-4c2c-9622-18ecdda402af tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:13.402629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5831ee19-2ef6-4c2c-9622-18ecdda402af tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:13.409032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5831ee19-2ef6-4c2c-9622-18ecdda402af tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:13.409527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5831ee19-2ef6-4c2c-9622-18ecdda402af tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:13.409967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5831ee19-2ef6-4c2c-9622-18ecdda402af tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:13.410768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5831ee19-2ef6-4c2c-9622-18ecdda402af tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:13.411221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5831ee19-2ef6-4c2c-9622-18ecdda402af tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:13.411654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5831ee19-2ef6-4c2c-9622-18ecdda402af tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:13.475576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5831ee19-2ef6-4c2c-9622-18ecdda402af tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:13.483422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5831ee19-2ef6-4c2c-9622-18ecdda402af tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.085521 Sep 08 13:58:13.483992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 320/552] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:13 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:14.189699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-760edb57-9d9d-4f0b-ba90-e9bdb652b8f4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:14.190839 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-760edb57-9d9d-4f0b-ba90-e9bdb652b8f4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:14.197568 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-760edb57-9d9d-4f0b-ba90-e9bdb652b8f4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:14.198085 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-760edb57-9d9d-4f0b-ba90-e9bdb652b8f4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:14.198581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-760edb57-9d9d-4f0b-ba90-e9bdb652b8f4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:14.199285 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-760edb57-9d9d-4f0b-ba90-e9bdb652b8f4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:14.199727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-760edb57-9d9d-4f0b-ba90-e9bdb652b8f4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:14.200145 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-760edb57-9d9d-4f0b-ba90-e9bdb652b8f4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:14.335501 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-760edb57-9d9d-4f0b-ba90-e9bdb652b8f4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:14.343620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-760edb57-9d9d-4f0b-ba90-e9bdb652b8f4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.158423 Sep 08 13:58:14.344190 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 321/553] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:14 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:14.423155 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f2635ed4-5f5f-4377-b833-3f152fe00999 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:14.423754 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f2635ed4-5f5f-4377-b833-3f152fe00999 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:14.429769 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2635ed4-5f5f-4377-b833-3f152fe00999 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:14.430291 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2635ed4-5f5f-4377-b833-3f152fe00999 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:14.430731 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2635ed4-5f5f-4377-b833-3f152fe00999 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:14.431429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2635ed4-5f5f-4377-b833-3f152fe00999 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:14.431880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2635ed4-5f5f-4377-b833-3f152fe00999 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:14.432306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2635ed4-5f5f-4377-b833-3f152fe00999 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:14.576368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f2635ed4-5f5f-4377-b833-3f152fe00999 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:14.587793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f2635ed4-5f5f-4377-b833-3f152fe00999 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:14.594389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f2635ed4-5f5f-4377-b833-3f152fe00999 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:14.594389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f2635ed4-5f5f-4377-b833-3f152fe00999 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.172223 Sep 08 13:58:14.594389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 322/554] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:14 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:14.596642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fa05da84-2f42-4683-8369-7c318910fe88 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:14.597250 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fa05da84-2f42-4683-8369-7c318910fe88 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:14.603478 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fa05da84-2f42-4683-8369-7c318910fe88 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:14.603931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fa05da84-2f42-4683-8369-7c318910fe88 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:14.604361 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fa05da84-2f42-4683-8369-7c318910fe88 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:14.605239 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fa05da84-2f42-4683-8369-7c318910fe88 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:14.605678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fa05da84-2f42-4683-8369-7c318910fe88 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:14.606105 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fa05da84-2f42-4683-8369-7c318910fe88 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:14.668553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fa05da84-2f42-4683-8369-7c318910fe88 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:14.676421 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fa05da84-2f42-4683-8369-7c318910fe88 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.083815 Sep 08 13:58:14.676984 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 323/555] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:14 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:15.366614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a5aa2e50-930a-4b7b-8732-1393da85a546 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:15.367361 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a5aa2e50-930a-4b7b-8732-1393da85a546 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:15.374792 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a5aa2e50-930a-4b7b-8732-1393da85a546 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:15.375275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a5aa2e50-930a-4b7b-8732-1393da85a546 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:15.375705 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a5aa2e50-930a-4b7b-8732-1393da85a546 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:15.376392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a5aa2e50-930a-4b7b-8732-1393da85a546 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:15.376842 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a5aa2e50-930a-4b7b-8732-1393da85a546 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:15.377260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a5aa2e50-930a-4b7b-8732-1393da85a546 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:15.511771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a5aa2e50-930a-4b7b-8732-1393da85a546 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:15.518940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a5aa2e50-930a-4b7b-8732-1393da85a546 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.156292 Sep 08 13:58:15.519514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 324/556] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:15 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:15.616382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8699adcf-18f7-46b2-ab4a-a112c3a63648 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:15.617068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8699adcf-18f7-46b2-ab4a-a112c3a63648 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:15.623094 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8699adcf-18f7-46b2-ab4a-a112c3a63648 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:15.623582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8699adcf-18f7-46b2-ab4a-a112c3a63648 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:15.624017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8699adcf-18f7-46b2-ab4a-a112c3a63648 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:15.624714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8699adcf-18f7-46b2-ab4a-a112c3a63648 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:15.625167 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8699adcf-18f7-46b2-ab4a-a112c3a63648 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:15.625647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8699adcf-18f7-46b2-ab4a-a112c3a63648 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:15.770903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8699adcf-18f7-46b2-ab4a-a112c3a63648 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:15.773087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:15.781142 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8699adcf-18f7-46b2-ab4a-a112c3a63648 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:15.785777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8699adcf-18f7-46b2-ab4a-a112c3a63648 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:15.785777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8699adcf-18f7-46b2-ab4a-a112c3a63648 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.171493 Sep 08 13:58:15.785777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 325/557] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:15 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:15.790091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-572a526a-682e-4c18-a2ca-17b72895ac3b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:15.790809 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-572a526a-682e-4c18-a2ca-17b72895ac3b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:15.797804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-572a526a-682e-4c18-a2ca-17b72895ac3b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:15.798322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-572a526a-682e-4c18-a2ca-17b72895ac3b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:15.798755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-572a526a-682e-4c18-a2ca-17b72895ac3b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:15.799609 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-572a526a-682e-4c18-a2ca-17b72895ac3b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:15.799933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-572a526a-682e-4c18-a2ca-17b72895ac3b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:15.800350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-572a526a-682e-4c18-a2ca-17b72895ac3b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:15.866759 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-572a526a-682e-4c18-a2ca-17b72895ac3b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:15.880713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-572a526a-682e-4c18-a2ca-17b72895ac3b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.090372 Sep 08 13:58:15.880713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 326/558] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:15 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:16.543288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-51ce69a4-b2a9-40f5-824b-1a2836b3e1c0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:16.544184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-51ce69a4-b2a9-40f5-824b-1a2836b3e1c0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:16.551169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51ce69a4-b2a9-40f5-824b-1a2836b3e1c0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:16.551669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51ce69a4-b2a9-40f5-824b-1a2836b3e1c0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:16.554028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51ce69a4-b2a9-40f5-824b-1a2836b3e1c0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:16.554028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51ce69a4-b2a9-40f5-824b-1a2836b3e1c0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:16.554028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51ce69a4-b2a9-40f5-824b-1a2836b3e1c0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:16.554028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51ce69a4-b2a9-40f5-824b-1a2836b3e1c0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:16.681966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-51ce69a4-b2a9-40f5-824b-1a2836b3e1c0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:16.690717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-51ce69a4-b2a9-40f5-824b-1a2836b3e1c0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.151772 Sep 08 13:58:16.691285 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 327/559] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:16 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:16.810267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8a5d3d0a-9964-4ff7-814f-13bdbcd9c15b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:16.810897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8a5d3d0a-9964-4ff7-814f-13bdbcd9c15b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:16.816778 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a5d3d0a-9964-4ff7-814f-13bdbcd9c15b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:16.817250 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a5d3d0a-9964-4ff7-814f-13bdbcd9c15b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:16.817740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a5d3d0a-9964-4ff7-814f-13bdbcd9c15b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:16.818398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a5d3d0a-9964-4ff7-814f-13bdbcd9c15b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:16.818845 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a5d3d0a-9964-4ff7-814f-13bdbcd9c15b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:16.819261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a5d3d0a-9964-4ff7-814f-13bdbcd9c15b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:16.951756 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8a5d3d0a-9964-4ff7-814f-13bdbcd9c15b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:16.960160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8a5d3d0a-9964-4ff7-814f-13bdbcd9c15b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:16.961616 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8a5d3d0a-9964-4ff7-814f-13bdbcd9c15b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:16.962769 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8a5d3d0a-9964-4ff7-814f-13bdbcd9c15b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.157047 Sep 08 13:58:16.963314 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 328/560] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:16 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:16.968216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fb8e51ca-d25e-4c31-b228-e7c3097a3ea4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:16.968802 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fb8e51ca-d25e-4c31-b228-e7c3097a3ea4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:16.974538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb8e51ca-d25e-4c31-b228-e7c3097a3ea4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:16.975018 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb8e51ca-d25e-4c31-b228-e7c3097a3ea4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:16.975448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb8e51ca-d25e-4c31-b228-e7c3097a3ea4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:16.976128 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb8e51ca-d25e-4c31-b228-e7c3097a3ea4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:16.976577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb8e51ca-d25e-4c31-b228-e7c3097a3ea4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:16.976993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb8e51ca-d25e-4c31-b228-e7c3097a3ea4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:17.037737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fb8e51ca-d25e-4c31-b228-e7c3097a3ea4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:17.045904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fb8e51ca-d25e-4c31-b228-e7c3097a3ea4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.081134 Sep 08 13:58:17.046504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 329/561] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:16 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:17.713178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0b54bdf2-aafb-4c9d-bbdb-365118701725 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:17.714344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0b54bdf2-aafb-4c9d-bbdb-365118701725 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:17.720796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b54bdf2-aafb-4c9d-bbdb-365118701725 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:17.721283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b54bdf2-aafb-4c9d-bbdb-365118701725 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:17.721951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b54bdf2-aafb-4c9d-bbdb-365118701725 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:17.722587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b54bdf2-aafb-4c9d-bbdb-365118701725 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:17.723046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b54bdf2-aafb-4c9d-bbdb-365118701725 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:17.723474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b54bdf2-aafb-4c9d-bbdb-365118701725 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:17.863983 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0b54bdf2-aafb-4c9d-bbdb-365118701725 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:17.871873 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0b54bdf2-aafb-4c9d-bbdb-365118701725 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.162519 Sep 08 13:58:17.872446 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 330/562] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:17 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:17.988966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1d23f320-26d7-44c3-aecc-348840a27201 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:17.989572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1d23f320-26d7-44c3-aecc-348840a27201 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:17.995686 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d23f320-26d7-44c3-aecc-348840a27201 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:17.996168 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d23f320-26d7-44c3-aecc-348840a27201 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:17.996606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d23f320-26d7-44c3-aecc-348840a27201 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:17.997295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d23f320-26d7-44c3-aecc-348840a27201 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:17.997755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d23f320-26d7-44c3-aecc-348840a27201 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:17.998233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d23f320-26d7-44c3-aecc-348840a27201 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:18.130032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1d23f320-26d7-44c3-aecc-348840a27201 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:18.139498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1d23f320-26d7-44c3-aecc-348840a27201 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:18.140964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1d23f320-26d7-44c3-aecc-348840a27201 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:18.142113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1d23f320-26d7-44c3-aecc-348840a27201 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.157147 Sep 08 13:58:18.142679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 331/563] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:17 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:18.147125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4120b165-6794-4b9f-ac95-b7093e994ac8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:18.147710 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4120b165-6794-4b9f-ac95-b7093e994ac8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:18.153813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4120b165-6794-4b9f-ac95-b7093e994ac8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:18.154422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4120b165-6794-4b9f-ac95-b7093e994ac8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:18.154857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4120b165-6794-4b9f-ac95-b7093e994ac8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:18.155803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4120b165-6794-4b9f-ac95-b7093e994ac8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:18.156254 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4120b165-6794-4b9f-ac95-b7093e994ac8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:18.156679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4120b165-6794-4b9f-ac95-b7093e994ac8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:18.218323 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4120b165-6794-4b9f-ac95-b7093e994ac8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:18.226918 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4120b165-6794-4b9f-ac95-b7093e994ac8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.082329 Sep 08 13:58:18.226918 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 332/564] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:18 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:18.894722 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0140741f-43bf-4c5c-a27d-8f859269ae1f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:18.895379 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0140741f-43bf-4c5c-a27d-8f859269ae1f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:18.901734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0140741f-43bf-4c5c-a27d-8f859269ae1f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:18.902258 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0140741f-43bf-4c5c-a27d-8f859269ae1f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:18.902702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0140741f-43bf-4c5c-a27d-8f859269ae1f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:18.903383 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0140741f-43bf-4c5c-a27d-8f859269ae1f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:18.903836 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0140741f-43bf-4c5c-a27d-8f859269ae1f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:18.904256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0140741f-43bf-4c5c-a27d-8f859269ae1f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:19.028987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0140741f-43bf-4c5c-a27d-8f859269ae1f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:19.036863 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0140741f-43bf-4c5c-a27d-8f859269ae1f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.146011 Sep 08 13:58:19.037424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 333/565] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:18 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:19.168276 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d970f137-b83b-43f6-998f-adccd36f175e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:19.168871 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d970f137-b83b-43f6-998f-adccd36f175e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:19.174815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d970f137-b83b-43f6-998f-adccd36f175e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:19.175297 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d970f137-b83b-43f6-998f-adccd36f175e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:19.175732 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d970f137-b83b-43f6-998f-adccd36f175e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:19.176421 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d970f137-b83b-43f6-998f-adccd36f175e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:19.176878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d970f137-b83b-43f6-998f-adccd36f175e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:19.177311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d970f137-b83b-43f6-998f-adccd36f175e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:19.307833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d970f137-b83b-43f6-998f-adccd36f175e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:19.316376 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d970f137-b83b-43f6-998f-adccd36f175e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:19.317802 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d970f137-b83b-43f6-998f-adccd36f175e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:19.318981 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d970f137-b83b-43f6-998f-adccd36f175e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.155052 Sep 08 13:58:19.319542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 334/566] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:19 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:19.323992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1647e8ce-6e67-4103-b4d5-97205e132985 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:19.324592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1647e8ce-6e67-4103-b4d5-97205e132985 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:19.330102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1647e8ce-6e67-4103-b4d5-97205e132985 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:19.330697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1647e8ce-6e67-4103-b4d5-97205e132985 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:19.331224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1647e8ce-6e67-4103-b4d5-97205e132985 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:19.331812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1647e8ce-6e67-4103-b4d5-97205e132985 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:19.332265 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1647e8ce-6e67-4103-b4d5-97205e132985 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:19.332690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1647e8ce-6e67-4103-b4d5-97205e132985 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:19.391957 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1647e8ce-6e67-4103-b4d5-97205e132985 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:19.399641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1647e8ce-6e67-4103-b4d5-97205e132985 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.079155 Sep 08 13:58:19.400203 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 335/567] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:19 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:20.060664 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-012d089e-e46c-4d6f-a8fd-76e08c4acca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:20.061650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-012d089e-e46c-4d6f-a8fd-76e08c4acca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:20.067801 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-012d089e-e46c-4d6f-a8fd-76e08c4acca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:20.068280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-012d089e-e46c-4d6f-a8fd-76e08c4acca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:20.068736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-012d089e-e46c-4d6f-a8fd-76e08c4acca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:20.069415 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-012d089e-e46c-4d6f-a8fd-76e08c4acca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:20.069865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-012d089e-e46c-4d6f-a8fd-76e08c4acca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:20.070318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-012d089e-e46c-4d6f-a8fd-76e08c4acca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:20.201400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-012d089e-e46c-4d6f-a8fd-76e08c4acca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:20.209078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-012d089e-e46c-4d6f-a8fd-76e08c4acca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.152769 Sep 08 13:58:20.209635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 336/568] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:20 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:20.344515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-88ac9cd1-c2ae-4bc3-ba64-dda725b6b57b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:20.345167 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-88ac9cd1-c2ae-4bc3-ba64-dda725b6b57b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:20.351052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-88ac9cd1-c2ae-4bc3-ba64-dda725b6b57b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:20.351527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-88ac9cd1-c2ae-4bc3-ba64-dda725b6b57b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:20.351954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-88ac9cd1-c2ae-4bc3-ba64-dda725b6b57b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:20.352628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-88ac9cd1-c2ae-4bc3-ba64-dda725b6b57b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:20.353106 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-88ac9cd1-c2ae-4bc3-ba64-dda725b6b57b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:20.353522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-88ac9cd1-c2ae-4bc3-ba64-dda725b6b57b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:20.482982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-88ac9cd1-c2ae-4bc3-ba64-dda725b6b57b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:20.491534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-88ac9cd1-c2ae-4bc3-ba64-dda725b6b57b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:20.492970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-88ac9cd1-c2ae-4bc3-ba64-dda725b6b57b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:20.494160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-88ac9cd1-c2ae-4bc3-ba64-dda725b6b57b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.153527 Sep 08 13:58:20.494716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 337/569] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:20 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:20.499703 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6a6743b3-9abf-473b-be78-d0988da108c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:20.500299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6a6743b3-9abf-473b-be78-d0988da108c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:20.505822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a6743b3-9abf-473b-be78-d0988da108c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:20.506341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a6743b3-9abf-473b-be78-d0988da108c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:20.506782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a6743b3-9abf-473b-be78-d0988da108c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:20.512702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a6743b3-9abf-473b-be78-d0988da108c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:20.512702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a6743b3-9abf-473b-be78-d0988da108c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:20.512702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a6743b3-9abf-473b-be78-d0988da108c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:20.568375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6a6743b3-9abf-473b-be78-d0988da108c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:20.575898 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6a6743b3-9abf-473b-be78-d0988da108c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.079597 Sep 08 13:58:20.576487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 338/570] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:20 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:21.231222 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2c16e403-fa98-4dcc-9c82-07bae0c6a674 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:21.231871 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2c16e403-fa98-4dcc-9c82-07bae0c6a674 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:21.238436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c16e403-fa98-4dcc-9c82-07bae0c6a674 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:21.238924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c16e403-fa98-4dcc-9c82-07bae0c6a674 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:21.239360 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c16e403-fa98-4dcc-9c82-07bae0c6a674 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:21.240068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c16e403-fa98-4dcc-9c82-07bae0c6a674 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:21.240508 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c16e403-fa98-4dcc-9c82-07bae0c6a674 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:21.240929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c16e403-fa98-4dcc-9c82-07bae0c6a674 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:21.394026 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2c16e403-fa98-4dcc-9c82-07bae0c6a674 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:21.402250 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2c16e403-fa98-4dcc-9c82-07bae0c6a674 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.174363 Sep 08 13:58:21.402807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 339/571] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:21 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:21.526657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-235d0438-e2fc-4c3a-bed5-ce4c61d329a3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:21.526657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-235d0438-e2fc-4c3a-bed5-ce4c61d329a3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:21.526657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-235d0438-e2fc-4c3a-bed5-ce4c61d329a3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:21.527062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-235d0438-e2fc-4c3a-bed5-ce4c61d329a3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:21.527351 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-235d0438-e2fc-4c3a-bed5-ce4c61d329a3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:21.528041 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-235d0438-e2fc-4c3a-bed5-ce4c61d329a3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:21.528497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-235d0438-e2fc-4c3a-bed5-ce4c61d329a3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:21.528923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-235d0438-e2fc-4c3a-bed5-ce4c61d329a3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:21.665746 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-235d0438-e2fc-4c3a-bed5-ce4c61d329a3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:21.677006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-235d0438-e2fc-4c3a-bed5-ce4c61d329a3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:21.678527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-235d0438-e2fc-4c3a-bed5-ce4c61d329a3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:21.679676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-235d0438-e2fc-4c3a-bed5-ce4c61d329a3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.163679 Sep 08 13:58:21.680242 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 340/572] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:21 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:21.696795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f01c2503-a55a-44ca-978b-125d18215da2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:21.696795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f01c2503-a55a-44ca-978b-125d18215da2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:21.696795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f01c2503-a55a-44ca-978b-125d18215da2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:21.696795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f01c2503-a55a-44ca-978b-125d18215da2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:21.697646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f01c2503-a55a-44ca-978b-125d18215da2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:21.697646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f01c2503-a55a-44ca-978b-125d18215da2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:21.697646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f01c2503-a55a-44ca-978b-125d18215da2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:21.697646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f01c2503-a55a-44ca-978b-125d18215da2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:21.759203 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f01c2503-a55a-44ca-978b-125d18215da2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:21.767372 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f01c2503-a55a-44ca-978b-125d18215da2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1138 microversion: 2.1 time: 0.086153 Sep 08 13:58:21.768391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 341/573] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:21 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1138 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:21.772910 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-af09518e-f19d-4f4e-a390-c45ab1c92f55 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:21.773399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-af09518e-f19d-4f4e-a390-c45ab1c92f55 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:21.779430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af09518e-f19d-4f4e-a390-c45ab1c92f55 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:21.779861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af09518e-f19d-4f4e-a390-c45ab1c92f55 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:21.780301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af09518e-f19d-4f4e-a390-c45ab1c92f55 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:21.781275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af09518e-f19d-4f4e-a390-c45ab1c92f55 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:21.781705 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af09518e-f19d-4f4e-a390-c45ab1c92f55 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:21.782128 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af09518e-f19d-4f4e-a390-c45ab1c92f55 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:21.797705 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:21.808344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-af09518e-f19d-4f4e-a390-c45ab1c92f55 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005/ips" status: 200 len: 150 microversion: 2.1 time: 0.038303 Sep 08 13:58:21.810177 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 342/574] 10.222.0.25 () {58 vars in 1248 bytes} [Thu Sep 8 13:58:21 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005/ips => generated 150 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:58:21.821025 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3230a51b-3c12-4d9e-bcc6-0a9ed58cfb38 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "06ab5239-fc50-4a85-aa7f-2e9dbf1b3e9d"}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:58:21.823874 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3230a51b-3c12-4d9e-bcc6-0a9ed58cfb38 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:21.830051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3230a51b-3c12-4d9e-bcc6-0a9ed58cfb38 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:21.830655 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3230a51b-3c12-4d9e-bcc6-0a9ed58cfb38 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:21.831096 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3230a51b-3c12-4d9e-bcc6-0a9ed58cfb38 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:21.831786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3230a51b-3c12-4d9e-bcc6-0a9ed58cfb38 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:21.832241 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3230a51b-3c12-4d9e-bcc6-0a9ed58cfb38 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:21.832669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3230a51b-3c12-4d9e-bcc6-0a9ed58cfb38 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:21.972854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-3230a51b-3c12-4d9e-bcc6-0a9ed58cfb38 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Invalid volume: volume 06ab5239-fc50-4a85-aa7f-2e9dbf1b3e9d is already attached to instances: 483c7eb7-3827-403f-84f3-c7c65dcbe574 Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3230a51b-3c12-4d9e-bcc6-0a9ed58cfb38 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: POST /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005/os-volume_attachments HTTP/1.1 Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 74 Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 13:58:21.974797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 13:58:21.976210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:58:21.976210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 13:58:21.976210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 13:58:21.976210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 13:58:21.976210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 13:58:21.976210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 13:58:21.976210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 13:58:21.976210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 13:58:21.976210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 13:58:21.976210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Sep 08 13:58:21.976210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {"volumeAttachment": {"volumeId": "06ab5239-fc50-4a85-aa7f-2e9dbf1b3e9d"}} Sep 08 13:58:21.976210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Invalid volume: volume 06ab5239-fc50-4a85-aa7f-2e9dbf1b3e9d is already attached to instances: 483c7eb7-3827-403f-84f3-c7c65dcbe574 Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'server_id': '82890cea-b89a-4020-b9aa-8a7d94ac6005', 'body': {'volumeAttachment': {'volumeId': '06ab5239-fc50-4a85-aa7f-2e9dbf1b3e9d'}}} Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 06ab5239-fc50-4a85-aa7f-2e9dbf1b3e9d is already attached to instances: 483c7eb7-3827-403f-84f3-c7c65dcbe574 Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:58:21.977000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 06ab5239-fc50-4a85-aa7f-2e9dbf1b3e9d is already attached to instances: 483c7eb7-3827-403f-84f3-c7c65dcbe574 Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:58:21.978652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:58:21.981163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:58:21.981163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:58:21.981163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:58:21.981163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 06ab5239-fc50-4a85-aa7f-2e9dbf1b3e9d is already attached to instances: 483c7eb7-3827-403f-84f3-c7c65dcbe574 Sep 08 13:58:21.981163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:58:21.981163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3230a51b-3c12-4d9e-bcc6-0a9ed58cfb38 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 400 to user: Invalid volume: volume 06ab5239-fc50-4a85-aa7f-2e9dbf1b3e9d is already attached to instances: 483c7eb7-3827-403f-84f3-c7c65dcbe574 {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:58:21.981163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3230a51b-3c12-4d9e-bcc6-0a9ed58cfb38 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "POST /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.159753 Sep 08 13:58:21.982083 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 343/575] 10.222.0.25 () {60 vars in 1323 bytes} [Thu Sep 8 13:58:21 2022] POST /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005/os-volume_attachments => generated 174 bytes in 166 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 08 13:58:21.987707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1aa09c15-3f5f-4fc6-ba46-653ab6c17e9c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:21.988300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1aa09c15-3f5f-4fc6-ba46-653ab6c17e9c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:21.995240 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1aa09c15-3f5f-4fc6-ba46-653ab6c17e9c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:21.996008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1aa09c15-3f5f-4fc6-ba46-653ab6c17e9c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:21.997211 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1aa09c15-3f5f-4fc6-ba46-653ab6c17e9c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:21.997947 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1aa09c15-3f5f-4fc6-ba46-653ab6c17e9c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:21.998964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1aa09c15-3f5f-4fc6-ba46-653ab6c17e9c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:21.998964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1aa09c15-3f5f-4fc6-ba46-653ab6c17e9c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:22.035681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1aa09c15-3f5f-4fc6-ba46-653ab6c17e9c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 13:58:22.279402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1aa09c15-3f5f-4fc6-ba46-653ab6c17e9c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 204 len: 0 microversion: 2.1 time: 0.295831 Sep 08 13:58:22.280681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 344/576] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 13:58:21 2022] DELETE /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 13:58:22.289506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1394c590-4cad-44ec-8374-2a75880d9c3e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:22.290237 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1394c590-4cad-44ec-8374-2a75880d9c3e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:22.296214 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1394c590-4cad-44ec-8374-2a75880d9c3e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:22.296697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1394c590-4cad-44ec-8374-2a75880d9c3e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:22.297133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1394c590-4cad-44ec-8374-2a75880d9c3e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:22.297824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1394c590-4cad-44ec-8374-2a75880d9c3e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:22.298396 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1394c590-4cad-44ec-8374-2a75880d9c3e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:22.298839 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1394c590-4cad-44ec-8374-2a75880d9c3e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:22.441357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1394c590-4cad-44ec-8374-2a75880d9c3e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:22.450614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1394c590-4cad-44ec-8374-2a75880d9c3e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1680 microversion: 2.1 time: 0.165376 Sep 08 13:58:22.451203 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 345/577] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:22 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1680 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:22.457226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2723fe76-6aab-43d2-9777-81ec42778454 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:22.458061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2723fe76-6aab-43d2-9777-81ec42778454 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:22.461893 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-79afc524-95f6-4573-b656-6be08f7dbae0 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "POST /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 31.225015 Sep 08 13:58:22.464018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 233/578] 10.222.0.25 () {60 vars in 1323 bytes} [Thu Sep 8 13:57:51 2022] POST /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments => generated 194 bytes in 31227 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:58:22.464883 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2723fe76-6aab-43d2-9777-81ec42778454 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:22.465381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2723fe76-6aab-43d2-9777-81ec42778454 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:22.465857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2723fe76-6aab-43d2-9777-81ec42778454 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:22.466602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2723fe76-6aab-43d2-9777-81ec42778454 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:22.467090 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2723fe76-6aab-43d2-9777-81ec42778454 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:22.467519 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2723fe76-6aab-43d2-9777-81ec42778454 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:22.639303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2723fe76-6aab-43d2-9777-81ec42778454 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:22.647382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2723fe76-6aab-43d2-9777-81ec42778454 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.194876 Sep 08 13:58:22.648007 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 346/579] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:22 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:22.710993 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-602a9825-eda7-48b2-880c-0701f6deabe7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:22.711720 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-602a9825-eda7-48b2-880c-0701f6deabe7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:22.718699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-602a9825-eda7-48b2-880c-0701f6deabe7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:22.719178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-602a9825-eda7-48b2-880c-0701f6deabe7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:22.719603 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-602a9825-eda7-48b2-880c-0701f6deabe7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:22.720289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-602a9825-eda7-48b2-880c-0701f6deabe7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:22.720738 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-602a9825-eda7-48b2-880c-0701f6deabe7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:22.721152 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-602a9825-eda7-48b2-880c-0701f6deabe7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:22.791365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b276893d-20d0-4628-96e3-bc158402f2c6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:22.792487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b276893d-20d0-4628-96e3-bc158402f2c6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:22.798762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b276893d-20d0-4628-96e3-bc158402f2c6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:22.799256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b276893d-20d0-4628-96e3-bc158402f2c6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:22.799696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b276893d-20d0-4628-96e3-bc158402f2c6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:22.800397 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b276893d-20d0-4628-96e3-bc158402f2c6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:22.800882 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b276893d-20d0-4628-96e3-bc158402f2c6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:22.801316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b276893d-20d0-4628-96e3-bc158402f2c6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:22.857997 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-602a9825-eda7-48b2-880c-0701f6deabe7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:22.863298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b276893d-20d0-4628-96e3-bc158402f2c6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:22.867444 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-602a9825-eda7-48b2-880c-0701f6deabe7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:22.868492 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-602a9825-eda7-48b2-880c-0701f6deabe7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:22.869682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-602a9825-eda7-48b2-880c-0701f6deabe7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.163027 Sep 08 13:58:22.870866 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 234/580] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:22 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:58:22.873099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b276893d-20d0-4628-96e3-bc158402f2c6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1186 microversion: 2.1 time: 0.085514 Sep 08 13:58:22.873663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 347/581] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:22 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1186 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:23.487352 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8457b4c0-94ba-4569-a9a4-fe939546896d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:23.487966 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8457b4c0-94ba-4569-a9a4-fe939546896d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:23.494488 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8457b4c0-94ba-4569-a9a4-fe939546896d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:23.495565 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8457b4c0-94ba-4569-a9a4-fe939546896d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:23.496008 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8457b4c0-94ba-4569-a9a4-fe939546896d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:23.496744 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8457b4c0-94ba-4569-a9a4-fe939546896d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:23.497246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8457b4c0-94ba-4569-a9a4-fe939546896d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:23.497663 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8457b4c0-94ba-4569-a9a4-fe939546896d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:23.633580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8457b4c0-94ba-4569-a9a4-fe939546896d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:23.641712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8457b4c0-94ba-4569-a9a4-fe939546896d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1680 microversion: 2.1 time: 0.158236 Sep 08 13:58:23.642340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 235/582] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:23 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1680 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:23.671488 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7fc92b01-a1c0-471b-9dbe-8f92fa4ea89c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:23.672112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7fc92b01-a1c0-471b-9dbe-8f92fa4ea89c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:23.679031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7fc92b01-a1c0-471b-9dbe-8f92fa4ea89c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:23.679672 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7fc92b01-a1c0-471b-9dbe-8f92fa4ea89c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:23.680113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7fc92b01-a1c0-471b-9dbe-8f92fa4ea89c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:23.680839 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7fc92b01-a1c0-471b-9dbe-8f92fa4ea89c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:23.681281 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7fc92b01-a1c0-471b-9dbe-8f92fa4ea89c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:23.681707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7fc92b01-a1c0-471b-9dbe-8f92fa4ea89c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:23.823679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7fc92b01-a1c0-471b-9dbe-8f92fa4ea89c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:23.831471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7fc92b01-a1c0-471b-9dbe-8f92fa4ea89c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.164421 Sep 08 13:58:23.832045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 348/583] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:23 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:23.896512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6b7b0e89-1ecc-41cd-b1ee-bf7e1e39708c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:23.896794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-91103fd2-f389-4f9b-b5c3-e7c84d23bcf1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:23.897116 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6b7b0e89-1ecc-41cd-b1ee-bf7e1e39708c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:23.897294 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-91103fd2-f389-4f9b-b5c3-e7c84d23bcf1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:23.903764 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6b7b0e89-1ecc-41cd-b1ee-bf7e1e39708c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:23.903957 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91103fd2-f389-4f9b-b5c3-e7c84d23bcf1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:23.904251 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6b7b0e89-1ecc-41cd-b1ee-bf7e1e39708c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:23.904437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91103fd2-f389-4f9b-b5c3-e7c84d23bcf1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:23.904679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6b7b0e89-1ecc-41cd-b1ee-bf7e1e39708c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:23.904785 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91103fd2-f389-4f9b-b5c3-e7c84d23bcf1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:23.905365 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6b7b0e89-1ecc-41cd-b1ee-bf7e1e39708c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:23.905482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91103fd2-f389-4f9b-b5c3-e7c84d23bcf1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:23.906078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91103fd2-f389-4f9b-b5c3-e7c84d23bcf1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:23.906233 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6b7b0e89-1ecc-41cd-b1ee-bf7e1e39708c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:23.906554 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6b7b0e89-1ecc-41cd-b1ee-bf7e1e39708c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:23.906657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91103fd2-f389-4f9b-b5c3-e7c84d23bcf1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:23.978529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-91103fd2-f389-4f9b-b5c3-e7c84d23bcf1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:23.987036 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-91103fd2-f389-4f9b-b5c3-e7c84d23bcf1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1186 microversion: 2.1 time: 0.094180 Sep 08 13:58:23.987829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 349/584] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:23 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1186 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:24.056226 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6b7b0e89-1ecc-41cd-b1ee-bf7e1e39708c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:24.067350 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6b7b0e89-1ecc-41cd-b1ee-bf7e1e39708c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:24.069195 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6b7b0e89-1ecc-41cd-b1ee-bf7e1e39708c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:24.070472 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6b7b0e89-1ecc-41cd-b1ee-bf7e1e39708c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.178051 Sep 08 13:58:24.071138 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 236/585] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:23 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:58:24.672603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2cf4b686-1baf-42dd-8bec-9a55a6b845f5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:24.673241 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2cf4b686-1baf-42dd-8bec-9a55a6b845f5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:24.679843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2cf4b686-1baf-42dd-8bec-9a55a6b845f5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:24.680345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2cf4b686-1baf-42dd-8bec-9a55a6b845f5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:24.680786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2cf4b686-1baf-42dd-8bec-9a55a6b845f5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:24.681687 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2cf4b686-1baf-42dd-8bec-9a55a6b845f5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:24.682171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2cf4b686-1baf-42dd-8bec-9a55a6b845f5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:24.682647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2cf4b686-1baf-42dd-8bec-9a55a6b845f5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:24.841583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2cf4b686-1baf-42dd-8bec-9a55a6b845f5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:24.856275 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8053186c-f8b2-4de0-bf7a-faf37ee718d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:24.857261 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8053186c-f8b2-4de0-bf7a-faf37ee718d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:24.876951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2cf4b686-1baf-42dd-8bec-9a55a6b845f5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1680 microversion: 2.1 time: 0.208027 Sep 08 13:58:24.877536 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 350/586] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:24 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1680 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:24.880431 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8053186c-f8b2-4de0-bf7a-faf37ee718d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:24.880926 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8053186c-f8b2-4de0-bf7a-faf37ee718d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:24.881361 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8053186c-f8b2-4de0-bf7a-faf37ee718d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:24.882248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8053186c-f8b2-4de0-bf7a-faf37ee718d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:24.887781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8053186c-f8b2-4de0-bf7a-faf37ee718d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:24.888259 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8053186c-f8b2-4de0-bf7a-faf37ee718d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:25.014518 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-058b62e7-c451-4b71-b07b-31e5678831ff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:25.015816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-058b62e7-c451-4b71-b07b-31e5678831ff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:25.024284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-058b62e7-c451-4b71-b07b-31e5678831ff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:25.027062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-058b62e7-c451-4b71-b07b-31e5678831ff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:25.027062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-058b62e7-c451-4b71-b07b-31e5678831ff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:25.027629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-058b62e7-c451-4b71-b07b-31e5678831ff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:25.028307 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-058b62e7-c451-4b71-b07b-31e5678831ff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:25.028949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-058b62e7-c451-4b71-b07b-31e5678831ff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:25.029613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8053186c-f8b2-4de0-bf7a-faf37ee718d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:25.039637 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8053186c-f8b2-4de0-bf7a-faf37ee718d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.188997 Sep 08 13:58:25.040217 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 237/587] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:24 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:25.103010 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3bf153d1-a359-4e65-883c-eadeeab19b74 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:25.103612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3bf153d1-a359-4e65-883c-eadeeab19b74 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:25.110261 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3bf153d1-a359-4e65-883c-eadeeab19b74 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:25.110748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3bf153d1-a359-4e65-883c-eadeeab19b74 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:25.111386 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3bf153d1-a359-4e65-883c-eadeeab19b74 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:25.112254 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3bf153d1-a359-4e65-883c-eadeeab19b74 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:25.112902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3bf153d1-a359-4e65-883c-eadeeab19b74 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:25.113731 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3bf153d1-a359-4e65-883c-eadeeab19b74 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:25.118083 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-058b62e7-c451-4b71-b07b-31e5678831ff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:25.138836 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-058b62e7-c451-4b71-b07b-31e5678831ff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1186 microversion: 2.1 time: 0.130985 Sep 08 13:58:25.139766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 351/588] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:25 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1186 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:25.280614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3bf153d1-a359-4e65-883c-eadeeab19b74 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:25.293398 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3bf153d1-a359-4e65-883c-eadeeab19b74 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:25.295554 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3bf153d1-a359-4e65-883c-eadeeab19b74 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:25.296718 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3bf153d1-a359-4e65-883c-eadeeab19b74 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.197615 Sep 08 13:58:25.297271 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 238/589] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:25 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 13:58:25.350243 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b24cea07-b766-4b6b-9e4b-21cecdfbe845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:58:25.352811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b24cea07-b766-4b6b-9e4b-21cecdfbe845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:25.360036 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b24cea07-b766-4b6b-9e4b-21cecdfbe845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:25.360640 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b24cea07-b766-4b6b-9e4b-21cecdfbe845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:25.360998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b24cea07-b766-4b6b-9e4b-21cecdfbe845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:25.362028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b24cea07-b766-4b6b-9e4b-21cecdfbe845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:25.362672 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b24cea07-b766-4b6b-9e4b-21cecdfbe845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:25.363329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b24cea07-b766-4b6b-9e4b-21cecdfbe845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:25.394591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-b24cea07-b766-4b6b-9e4b-21cecdfbe845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lazy-loading 'flavor' on Instance uuid 19db4aa2-f574-43fa-bbf8-c3042b243152 {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 13:58:25.790163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b24cea07-b766-4b6b-9e4b-21cecdfbe845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "POST /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/action" status: 200 len: 32 microversion: 2.1 time: 0.443926 Sep 08 13:58:25.791195 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 352/590] 10.222.0.25 () {60 vars in 1278 bytes} [Thu Sep 8 13:58:25 2022] POST /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/action => generated 32 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 08 13:58:25.804261 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-10c9e0a9-54c0-429e-8fb7-7faf4e038d23 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:25.804926 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-10c9e0a9-54c0-429e-8fb7-7faf4e038d23 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:25.813902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-10c9e0a9-54c0-429e-8fb7-7faf4e038d23 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:25.814771 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-10c9e0a9-54c0-429e-8fb7-7faf4e038d23 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:25.815343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-10c9e0a9-54c0-429e-8fb7-7faf4e038d23 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:25.816374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-10c9e0a9-54c0-429e-8fb7-7faf4e038d23 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:25.817026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-10c9e0a9-54c0-429e-8fb7-7faf4e038d23 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:25.817638 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-10c9e0a9-54c0-429e-8fb7-7faf4e038d23 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:25.902423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8228cb62-5999-4bb3-92d8-ea460d81cd60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:25.903112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8228cb62-5999-4bb3-92d8-ea460d81cd60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:25.905028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:25.910350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8228cb62-5999-4bb3-92d8-ea460d81cd60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:25.913669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8228cb62-5999-4bb3-92d8-ea460d81cd60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:25.913669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8228cb62-5999-4bb3-92d8-ea460d81cd60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:25.913669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8228cb62-5999-4bb3-92d8-ea460d81cd60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:25.913669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8228cb62-5999-4bb3-92d8-ea460d81cd60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:25.914233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8228cb62-5999-4bb3-92d8-ea460d81cd60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:25.955177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-10c9e0a9-54c0-429e-8fb7-7faf4e038d23 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:25.964320 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-10c9e0a9-54c0-429e-8fb7-7faf4e038d23 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.165747 Sep 08 13:58:25.964932 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 239/591] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:25 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:26.052137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8228cb62-5999-4bb3-92d8-ea460d81cd60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:26.060760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8228cb62-5999-4bb3-92d8-ea460d81cd60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1680 microversion: 2.1 time: 0.163213 Sep 08 13:58:26.063836 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4efd5115-c187-4edd-8d29-a55935576105 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:26.064445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4efd5115-c187-4edd-8d29-a55935576105 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:26.070977 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4efd5115-c187-4edd-8d29-a55935576105 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:26.071449 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4efd5115-c187-4edd-8d29-a55935576105 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:26.078543 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4efd5115-c187-4edd-8d29-a55935576105 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:26.078543 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4efd5115-c187-4edd-8d29-a55935576105 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:26.078543 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4efd5115-c187-4edd-8d29-a55935576105 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:26.078543 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4efd5115-c187-4edd-8d29-a55935576105 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:26.078947 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 353/592] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:25 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1680 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:26.162225 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f90bcaf7-ed71-4089-a48b-68b34f5ac8d0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:26.162832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f90bcaf7-ed71-4089-a48b-68b34f5ac8d0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:26.168850 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f90bcaf7-ed71-4089-a48b-68b34f5ac8d0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:26.169341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f90bcaf7-ed71-4089-a48b-68b34f5ac8d0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:26.169775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f90bcaf7-ed71-4089-a48b-68b34f5ac8d0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:26.170515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f90bcaf7-ed71-4089-a48b-68b34f5ac8d0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:26.170967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f90bcaf7-ed71-4089-a48b-68b34f5ac8d0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:26.171391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f90bcaf7-ed71-4089-a48b-68b34f5ac8d0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:26.228663 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4efd5115-c187-4edd-8d29-a55935576105 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:26.237059 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4efd5115-c187-4edd-8d29-a55935576105 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.177132 Sep 08 13:58:26.237630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 240/593] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:26 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:26.242253 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f90bcaf7-ed71-4089-a48b-68b34f5ac8d0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:26.253084 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f90bcaf7-ed71-4089-a48b-68b34f5ac8d0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1186 microversion: 2.1 time: 0.094604 Sep 08 13:58:26.253666 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 354/594] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:26 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1186 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:26.322458 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6b55ee33-9605-4755-92b7-2aa5d6c9af2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:26.323060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6b55ee33-9605-4755-92b7-2aa5d6c9af2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:26.329567 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6b55ee33-9605-4755-92b7-2aa5d6c9af2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:26.330052 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6b55ee33-9605-4755-92b7-2aa5d6c9af2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:26.330529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6b55ee33-9605-4755-92b7-2aa5d6c9af2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:26.331213 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6b55ee33-9605-4755-92b7-2aa5d6c9af2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:26.331666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6b55ee33-9605-4755-92b7-2aa5d6c9af2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:26.332086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6b55ee33-9605-4755-92b7-2aa5d6c9af2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:26.477947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6b55ee33-9605-4755-92b7-2aa5d6c9af2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:26.487535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6b55ee33-9605-4755-92b7-2aa5d6c9af2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:26.488984 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6b55ee33-9605-4755-92b7-2aa5d6c9af2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:26.490286 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6b55ee33-9605-4755-92b7-2aa5d6c9af2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.171811 Sep 08 13:58:26.491269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 241/595] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:26 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:58:26.852492 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-55b59170-222c-4636-a3cc-42d8e97fd0ba req-165fc12e-b196-4df7-8040-df7bba0a0196 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "82890cea-b89a-4020-b9aa-8a7d94ac6005", "name": "network-vif-unplugged", "status": "completed", "tag": "c152a3b2-3cdc-424f-9020-7dba12d0cb18"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:58:26.865928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-55b59170-222c-4636-a3cc-42d8e97fd0ba req-165fc12e-b196-4df7-8040-df7bba0a0196 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:26.867884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-55b59170-222c-4636-a3cc-42d8e97fd0ba req-165fc12e-b196-4df7-8040-df7bba0a0196 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:26.867884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-55b59170-222c-4636-a3cc-42d8e97fd0ba req-165fc12e-b196-4df7-8040-df7bba0a0196 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:26.884854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-55b59170-222c-4636-a3cc-42d8e97fd0ba req-165fc12e-b196-4df7-8040-df7bba0a0196 service nova] Creating event network-vif-unplugged:c152a3b2-3cdc-424f-9020-7dba12d0cb18 for instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 on hv-cinder-85646 Sep 08 13:58:26.891037 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-55b59170-222c-4636-a3cc-42d8e97fd0ba req-165fc12e-b196-4df7-8040-df7bba0a0196 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.042376 Sep 08 13:58:26.891942 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 355/596] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:58:26 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:58:26.988600 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8e44764e-e31b-46b6-b256-92cf307d34d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:26.989154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8e44764e-e31b-46b6-b256-92cf307d34d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:26.999444 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8e44764e-e31b-46b6-b256-92cf307d34d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:27.000715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8e44764e-e31b-46b6-b256-92cf307d34d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:27.001895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8e44764e-e31b-46b6-b256-92cf307d34d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:27.003908 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8e44764e-e31b-46b6-b256-92cf307d34d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:27.004988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8e44764e-e31b-46b6-b256-92cf307d34d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:27.006577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8e44764e-e31b-46b6-b256-92cf307d34d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:27.090307 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-232bc70f-e501-4861-b1da-f5da4b9357ec tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:27.091338 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-232bc70f-e501-4861-b1da-f5da4b9357ec tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:27.100011 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-232bc70f-e501-4861-b1da-f5da4b9357ec tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:27.100804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-232bc70f-e501-4861-b1da-f5da4b9357ec tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:27.101509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-232bc70f-e501-4861-b1da-f5da4b9357ec tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:27.102479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-232bc70f-e501-4861-b1da-f5da4b9357ec tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:27.103008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-232bc70f-e501-4861-b1da-f5da4b9357ec tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:27.103512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-232bc70f-e501-4861-b1da-f5da4b9357ec tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:27.121440 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:27.268925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-232bc70f-e501-4861-b1da-f5da4b9357ec tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:27.271135 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8e44764e-e31b-46b6-b256-92cf307d34d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:27.286823 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-232bc70f-e501-4861-b1da-f5da4b9357ec tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1680 microversion: 2.1 time: 0.201042 Sep 08 13:58:27.288621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 356/597] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:27 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1680 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:27.290067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8e44764e-e31b-46b6-b256-92cf307d34d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.305786 Sep 08 13:58:27.291310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 242/598] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:26 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:27.294192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0782b691-e425-4a16-9ba7-44fe03569711 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:27.294870 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0782b691-e425-4a16-9ba7-44fe03569711 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:27.296806 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-25082f9d-5a9a-4658-a92f-5ee60f990fc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:27.297591 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-25082f9d-5a9a-4658-a92f-5ee60f990fc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:27.303399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0782b691-e425-4a16-9ba7-44fe03569711 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:27.303899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0782b691-e425-4a16-9ba7-44fe03569711 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:27.304340 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0782b691-e425-4a16-9ba7-44fe03569711 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:27.305550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-25082f9d-5a9a-4658-a92f-5ee60f990fc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:27.305713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0782b691-e425-4a16-9ba7-44fe03569711 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:27.306163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0782b691-e425-4a16-9ba7-44fe03569711 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:27.306656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-25082f9d-5a9a-4658-a92f-5ee60f990fc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:27.306950 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0782b691-e425-4a16-9ba7-44fe03569711 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:27.307440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-25082f9d-5a9a-4658-a92f-5ee60f990fc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:27.308353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-25082f9d-5a9a-4658-a92f-5ee60f990fc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:27.309164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-25082f9d-5a9a-4658-a92f-5ee60f990fc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:27.309872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-25082f9d-5a9a-4658-a92f-5ee60f990fc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:27.487267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0782b691-e425-4a16-9ba7-44fe03569711 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:27.496021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0782b691-e425-4a16-9ba7-44fe03569711 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.206481 Sep 08 13:58:27.496944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 357/599] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:27 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:27.525201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c0915c13-753e-4a26-b738-1ff48df4dd85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:27.526308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c0915c13-753e-4a26-b738-1ff48df4dd85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:27.533683 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0915c13-753e-4a26-b738-1ff48df4dd85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:27.534499 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0915c13-753e-4a26-b738-1ff48df4dd85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:27.534948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0915c13-753e-4a26-b738-1ff48df4dd85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:27.535703 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0915c13-753e-4a26-b738-1ff48df4dd85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:27.536161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0915c13-753e-4a26-b738-1ff48df4dd85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:27.536590 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0915c13-753e-4a26-b738-1ff48df4dd85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:27.684795 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-25082f9d-5a9a-4658-a92f-5ee60f990fc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:27.695111 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-25082f9d-5a9a-4658-a92f-5ee60f990fc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1186 microversion: 2.1 time: 0.402582 Sep 08 13:58:27.695813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 243/600] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:27 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1186 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:27.737350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c0915c13-753e-4a26-b738-1ff48df4dd85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:27.749041 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c0915c13-753e-4a26-b738-1ff48df4dd85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:27.752749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c0915c13-753e-4a26-b738-1ff48df4dd85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:27.754285 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c0915c13-753e-4a26-b738-1ff48df4dd85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.233502 Sep 08 13:58:27.755255 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 358/601] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:27 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:58:28.331001 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-89ef4b5a-85a8-4feb-84a2-8705dd436e7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:28.337679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-be3b3175-535c-4854-9a0e-d98ef1df81ed tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:28.339893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-89ef4b5a-85a8-4feb-84a2-8705dd436e7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:28.342477 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-be3b3175-535c-4854-9a0e-d98ef1df81ed tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:28.351122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be3b3175-535c-4854-9a0e-d98ef1df81ed tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:28.351420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-89ef4b5a-85a8-4feb-84a2-8705dd436e7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:28.351913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-89ef4b5a-85a8-4feb-84a2-8705dd436e7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:28.352351 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-89ef4b5a-85a8-4feb-84a2-8705dd436e7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:28.352475 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be3b3175-535c-4854-9a0e-d98ef1df81ed tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:28.353033 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be3b3175-535c-4854-9a0e-d98ef1df81ed tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:28.353182 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-89ef4b5a-85a8-4feb-84a2-8705dd436e7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:28.353484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-89ef4b5a-85a8-4feb-84a2-8705dd436e7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:28.354098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-89ef4b5a-85a8-4feb-84a2-8705dd436e7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:28.354352 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be3b3175-535c-4854-9a0e-d98ef1df81ed tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:28.356586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be3b3175-535c-4854-9a0e-d98ef1df81ed tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:28.357818 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be3b3175-535c-4854-9a0e-d98ef1df81ed tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:28.508651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-be3b3175-535c-4854-9a0e-d98ef1df81ed tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:28.518945 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-be3b3175-535c-4854-9a0e-d98ef1df81ed tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1612 microversion: 2.1 time: 0.185824 Sep 08 13:58:28.519927 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 244/602] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:28 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1612 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:28.527048 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-67da2ad2-8a73-4561-8a6c-f73a2edfc3b6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:28.527709 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-89ef4b5a-85a8-4feb-84a2-8705dd436e7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:28.527939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-67da2ad2-8a73-4561-8a6c-f73a2edfc3b6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:28.538108 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-89ef4b5a-85a8-4feb-84a2-8705dd436e7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.210994 Sep 08 13:58:28.539628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 359/603] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:28 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:28.545405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67da2ad2-8a73-4561-8a6c-f73a2edfc3b6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:28.545405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67da2ad2-8a73-4561-8a6c-f73a2edfc3b6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:28.545405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67da2ad2-8a73-4561-8a6c-f73a2edfc3b6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:28.545752 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67da2ad2-8a73-4561-8a6c-f73a2edfc3b6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:28.546094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67da2ad2-8a73-4561-8a6c-f73a2edfc3b6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:28.546611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67da2ad2-8a73-4561-8a6c-f73a2edfc3b6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:28.722751 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4fbea45a-741d-4d17-80e4-63d709b18413 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:28.723723 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4fbea45a-741d-4d17-80e4-63d709b18413 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:28.730362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fbea45a-741d-4d17-80e4-63d709b18413 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:28.738231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fbea45a-741d-4d17-80e4-63d709b18413 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:28.738231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fbea45a-741d-4d17-80e4-63d709b18413 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:28.738231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fbea45a-741d-4d17-80e4-63d709b18413 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:28.738231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fbea45a-741d-4d17-80e4-63d709b18413 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:28.738694 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fbea45a-741d-4d17-80e4-63d709b18413 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:28.803430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4fbea45a-741d-4d17-80e4-63d709b18413 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:28.814539 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4fbea45a-741d-4d17-80e4-63d709b18413 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1194 microversion: 2.1 time: 0.095514 Sep 08 13:58:28.830175 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 360/604] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:28 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1194 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:28.830175 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e4c077ed-abe6-4b28-a82a-b4b2fb0475e3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:28.830175 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e4c077ed-abe6-4b28-a82a-b4b2fb0475e3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:28.832035 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e4c077ed-abe6-4b28-a82a-b4b2fb0475e3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:28.833516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e4c077ed-abe6-4b28-a82a-b4b2fb0475e3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:28.834195 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e4c077ed-abe6-4b28-a82a-b4b2fb0475e3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:28.835660 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e4c077ed-abe6-4b28-a82a-b4b2fb0475e3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:28.836102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e4c077ed-abe6-4b28-a82a-b4b2fb0475e3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:28.837605 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e4c077ed-abe6-4b28-a82a-b4b2fb0475e3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:29.086524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e4c077ed-abe6-4b28-a82a-b4b2fb0475e3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:29.097708 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e4c077ed-abe6-4b28-a82a-b4b2fb0475e3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:29.100100 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e4c077ed-abe6-4b28-a82a-b4b2fb0475e3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:29.100784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-67da2ad2-8a73-4561-8a6c-f73a2edfc3b6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:29.102334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e4c077ed-abe6-4b28-a82a-b4b2fb0475e3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1807 microversion: 2.20 time: 0.285285 Sep 08 13:58:29.108829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-67da2ad2-8a73-4561-8a6c-f73a2edfc3b6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.586312 Sep 08 13:58:29.109420 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 245/605] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:28 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 588 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:29.114290 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 361/606] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:28 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1807 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:58:29.161099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-671a9282-671b-425e-8f91-ce3bf921ee20 req-94f0877d-280e-4af4-a9af-d3b2f6f9a4f3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "82890cea-b89a-4020-b9aa-8a7d94ac6005", "name": "network-vif-deleted", "tag": "c152a3b2-3cdc-424f-9020-7dba12d0cb18"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:58:29.171262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-671a9282-671b-425e-8f91-ce3bf921ee20 req-94f0877d-280e-4af4-a9af-d3b2f6f9a4f3 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:29.171881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-671a9282-671b-425e-8f91-ce3bf921ee20 req-94f0877d-280e-4af4-a9af-d3b2f6f9a4f3 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:29.172541 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-671a9282-671b-425e-8f91-ce3bf921ee20 req-94f0877d-280e-4af4-a9af-d3b2f6f9a4f3 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:29.185488 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-671a9282-671b-425e-8f91-ce3bf921ee20 req-94f0877d-280e-4af4-a9af-d3b2f6f9a4f3 service nova] Creating event network-vif-deleted:c152a3b2-3cdc-424f-9020-7dba12d0cb18 for instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 on hv-cinder-85646 Sep 08 13:58:29.192540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-671a9282-671b-425e-8f91-ce3bf921ee20 req-94f0877d-280e-4af4-a9af-d3b2f6f9a4f3 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034575 Sep 08 13:58:29.193469 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 246/607] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:58:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:58:29.559907 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-60e98b5e-4ad8-4a10-9869-20eaa93f7503 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:29.561400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-60e98b5e-4ad8-4a10-9869-20eaa93f7503 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:29.571008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-60e98b5e-4ad8-4a10-9869-20eaa93f7503 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:29.577122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-60e98b5e-4ad8-4a10-9869-20eaa93f7503 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:29.577122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-60e98b5e-4ad8-4a10-9869-20eaa93f7503 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:29.577122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-60e98b5e-4ad8-4a10-9869-20eaa93f7503 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:29.577122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-60e98b5e-4ad8-4a10-9869-20eaa93f7503 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:29.577567 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-653f015e-a372-4c64-b708-a55541228603 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:29.577994 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-60e98b5e-4ad8-4a10-9869-20eaa93f7503 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:29.578091 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-653f015e-a372-4c64-b708-a55541228603 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:29.586009 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-653f015e-a372-4c64-b708-a55541228603 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:29.586981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-653f015e-a372-4c64-b708-a55541228603 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:29.587601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-653f015e-a372-4c64-b708-a55541228603 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:29.593932 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-653f015e-a372-4c64-b708-a55541228603 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:29.593932 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-653f015e-a372-4c64-b708-a55541228603 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:29.593932 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-653f015e-a372-4c64-b708-a55541228603 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:29.861808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-653f015e-a372-4c64-b708-a55541228603 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:29.883392 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-653f015e-a372-4c64-b708-a55541228603 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.319581 Sep 08 13:58:29.900664 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 247/608] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:29 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:29.922346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-38c546ab-691f-4ec2-b3b9-b105eb4efa84 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:29.922987 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-38c546ab-691f-4ec2-b3b9-b105eb4efa84 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:29.939423 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38c546ab-691f-4ec2-b3b9-b105eb4efa84 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:29.939982 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38c546ab-691f-4ec2-b3b9-b105eb4efa84 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:29.940413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38c546ab-691f-4ec2-b3b9-b105eb4efa84 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:29.941852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38c546ab-691f-4ec2-b3b9-b105eb4efa84 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:29.942494 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38c546ab-691f-4ec2-b3b9-b105eb4efa84 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:29.943801 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38c546ab-691f-4ec2-b3b9-b105eb4efa84 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:30.067870 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-60e98b5e-4ad8-4a10-9869-20eaa93f7503 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:30.077264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-60e98b5e-4ad8-4a10-9869-20eaa93f7503 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1324 microversion: 2.1 time: 0.522515 Sep 08 13:58:30.078260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 362/609] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:29 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1324 bytes in 524 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:30.100551 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-38c546ab-691f-4ec2-b3b9-b105eb4efa84 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:30.111906 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-38c546ab-691f-4ec2-b3b9-b105eb4efa84 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1204 microversion: 2.1 time: 0.210034 Sep 08 13:58:30.112489 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 248/610] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:29 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1204 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:30.139685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4c22ba16-7fee-462c-869e-c9848f64ae98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:30.140484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4c22ba16-7fee-462c-869e-c9848f64ae98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:30.154725 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fb5b4aa4-1e9a-4cba-a87e-7d4bd295dae8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:30.156947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fb5b4aa4-1e9a-4cba-a87e-7d4bd295dae8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:30.163215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4c22ba16-7fee-462c-869e-c9848f64ae98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:30.164661 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb5b4aa4-1e9a-4cba-a87e-7d4bd295dae8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:30.165718 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb5b4aa4-1e9a-4cba-a87e-7d4bd295dae8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:30.166404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb5b4aa4-1e9a-4cba-a87e-7d4bd295dae8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:30.167319 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb5b4aa4-1e9a-4cba-a87e-7d4bd295dae8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:30.168136 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4c22ba16-7fee-462c-869e-c9848f64ae98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:30.168357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb5b4aa4-1e9a-4cba-a87e-7d4bd295dae8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:30.168966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4c22ba16-7fee-462c-869e-c9848f64ae98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:30.169298 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb5b4aa4-1e9a-4cba-a87e-7d4bd295dae8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:30.169706 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4c22ba16-7fee-462c-869e-c9848f64ae98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:30.170190 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4c22ba16-7fee-462c-869e-c9848f64ae98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:30.170624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4c22ba16-7fee-462c-869e-c9848f64ae98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:30.381229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4c22ba16-7fee-462c-869e-c9848f64ae98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:30.386761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-fb5b4aa4-1e9a-4cba-a87e-7d4bd295dae8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:30.391734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:30.395484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4c22ba16-7fee-462c-869e-c9848f64ae98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:30.397764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4c22ba16-7fee-462c-869e-c9848f64ae98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:30.399494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4c22ba16-7fee-462c-869e-c9848f64ae98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 200 len: 1794 microversion: 2.20 time: 0.274930 Sep 08 13:58:30.400556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 363/611] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 13:58:30 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 1794 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 13:58:30.406602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fb5b4aa4-1e9a-4cba-a87e-7d4bd295dae8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.254456 Sep 08 13:58:30.412510 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 249/612] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:30 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:30.926403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a4ccb5c4-1f61-4e0b-9487-eda697e0b2ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:30.928307 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a4ccb5c4-1f61-4e0b-9487-eda697e0b2ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:30.941916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4ccb5c4-1f61-4e0b-9487-eda697e0b2ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:30.942690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4ccb5c4-1f61-4e0b-9487-eda697e0b2ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:30.943173 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4ccb5c4-1f61-4e0b-9487-eda697e0b2ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:30.945487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4ccb5c4-1f61-4e0b-9487-eda697e0b2ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:30.946196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4ccb5c4-1f61-4e0b-9487-eda697e0b2ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:30.946820 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4ccb5c4-1f61-4e0b-9487-eda697e0b2ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:31.098342 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a4ccb5c4-1f61-4e0b-9487-eda697e0b2ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:31.106017 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a84ace1c-31ef-4d7f-8eb9-e66a7e6dc323 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:31.109630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a84ace1c-31ef-4d7f-8eb9-e66a7e6dc323 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:31.119932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a4ccb5c4-1f61-4e0b-9487-eda697e0b2ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.208778 Sep 08 13:58:31.130180 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a84ace1c-31ef-4d7f-8eb9-e66a7e6dc323 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:31.131062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 364/613] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:30 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:31.132138 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a84ace1c-31ef-4d7f-8eb9-e66a7e6dc323 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:31.135957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a84ace1c-31ef-4d7f-8eb9-e66a7e6dc323 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:31.137566 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a84ace1c-31ef-4d7f-8eb9-e66a7e6dc323 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:31.143649 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a84ace1c-31ef-4d7f-8eb9-e66a7e6dc323 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:31.144806 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a84ace1c-31ef-4d7f-8eb9-e66a7e6dc323 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:31.145695 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-58fdac86-949c-4ef9-b6ad-e830bd749bef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:31.153698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-58fdac86-949c-4ef9-b6ad-e830bd749bef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:31.160809 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58fdac86-949c-4ef9-b6ad-e830bd749bef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:31.161333 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58fdac86-949c-4ef9-b6ad-e830bd749bef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:31.161789 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58fdac86-949c-4ef9-b6ad-e830bd749bef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:31.162545 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58fdac86-949c-4ef9-b6ad-e830bd749bef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:31.163007 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58fdac86-949c-4ef9-b6ad-e830bd749bef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:31.163471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58fdac86-949c-4ef9-b6ad-e830bd749bef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:31.187461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a84ace1c-31ef-4d7f-8eb9-e66a7e6dc323 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:31.188214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a84ace1c-31ef-4d7f-8eb9-e66a7e6dc323 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:31.188853 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a84ace1c-31ef-4d7f-8eb9-e66a7e6dc323 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:31.189466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-a84ace1c-31ef-4d7f-8eb9-e66a7e6dc323 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lazy-loading 'fault' on Instance uuid 82890cea-b89a-4020-b9aa-8a7d94ac6005 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 13:58:31.344783 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a84ace1c-31ef-4d7f-8eb9-e66a7e6dc323 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:31.354198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a84ace1c-31ef-4d7f-8eb9-e66a7e6dc323 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 200 len: 1329 microversion: 2.1 time: 0.253942 Sep 08 13:58:31.355849 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 250/614] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:31 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 1329 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:31.362513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-58fdac86-949c-4ef9-b6ad-e830bd749bef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:31.371429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-58fdac86-949c-4ef9-b6ad-e830bd749bef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1272 microversion: 2.1 time: 0.235690 Sep 08 13:58:31.372118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 365/615] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:31 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1272 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:31.435303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6200f432-b91b-4618-b4ce-dbb827bd365b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:31.435303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6200f432-b91b-4618-b4ce-dbb827bd365b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:31.443583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6200f432-b91b-4618-b4ce-dbb827bd365b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:31.444288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6200f432-b91b-4618-b4ce-dbb827bd365b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:31.444760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6200f432-b91b-4618-b4ce-dbb827bd365b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:31.445466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6200f432-b91b-4618-b4ce-dbb827bd365b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:31.446023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6200f432-b91b-4618-b4ce-dbb827bd365b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:31.446479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6200f432-b91b-4618-b4ce-dbb827bd365b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:31.910714 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6200f432-b91b-4618-b4ce-dbb827bd365b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:31.923376 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6200f432-b91b-4618-b4ce-dbb827bd365b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.493526 Sep 08 13:58:31.927987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-43f78d32-c015-45ce-af50-508e4bfc3381 req-8f49572a-0daa-4492-addf-a69e6a6213a9 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d7ae43e4-11c8-417c-be73-4f9fb5c42f2c", "name": "network-vif-plugged", "status": "completed", "tag": "48757fc0-370f-4fb9-a0d2-0688efed54e8"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:58:31.935297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 251/616] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:31 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:31.939853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-43f78d32-c015-45ce-af50-508e4bfc3381 req-8f49572a-0daa-4492-addf-a69e6a6213a9 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:31.940652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-43f78d32-c015-45ce-af50-508e4bfc3381 req-8f49572a-0daa-4492-addf-a69e6a6213a9 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:31.941449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-43f78d32-c015-45ce-af50-508e4bfc3381 req-8f49572a-0daa-4492-addf-a69e6a6213a9 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:31.953908 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-43f78d32-c015-45ce-af50-508e4bfc3381 req-8f49572a-0daa-4492-addf-a69e6a6213a9 service nova] Creating event network-vif-plugged:48757fc0-370f-4fb9-a0d2-0688efed54e8 for instance d7ae43e4-11c8-417c-be73-4f9fb5c42f2c on hv-cinder-85646 Sep 08 13:58:31.958999 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-43f78d32-c015-45ce-af50-508e4bfc3381 req-8f49572a-0daa-4492-addf-a69e6a6213a9 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036312 Sep 08 13:58:31.959987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 366/617] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:58:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:58:32.132067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-fada06f5-3fdd-44e3-916f-987fd14fb3bb req-d04de9e8-73fd-49da-93a7-7f7df2a6541a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d1062abf-7525-48c5-a9cb-91da3855fd1b", "tag": "2d04f3d2-99c9-4712-bc13-6ebdffcb949f"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:58:32.143734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-fada06f5-3fdd-44e3-916f-987fd14fb3bb req-d04de9e8-73fd-49da-93a7-7f7df2a6541a service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:32.144234 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-fada06f5-3fdd-44e3-916f-987fd14fb3bb req-d04de9e8-73fd-49da-93a7-7f7df2a6541a service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:32.146291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-fada06f5-3fdd-44e3-916f-987fd14fb3bb req-d04de9e8-73fd-49da-93a7-7f7df2a6541a service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:32.155302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6a2a241a-37ed-4c66-b9d9-9f3980289c96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:32.156600 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6a2a241a-37ed-4c66-b9d9-9f3980289c96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:32.159982 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-fada06f5-3fdd-44e3-916f-987fd14fb3bb req-d04de9e8-73fd-49da-93a7-7f7df2a6541a service nova] Creating event network-changed:2d04f3d2-99c9-4712-bc13-6ebdffcb949f for instance d1062abf-7525-48c5-a9cb-91da3855fd1b on hv-cinder-85646 Sep 08 13:58:32.166476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a2a241a-37ed-4c66-b9d9-9f3980289c96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:32.167006 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-fada06f5-3fdd-44e3-916f-987fd14fb3bb req-d04de9e8-73fd-49da-93a7-7f7df2a6541a service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.038455 Sep 08 13:58:32.168481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a2a241a-37ed-4c66-b9d9-9f3980289c96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:32.169228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 252/618] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:58:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:58:32.169410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a2a241a-37ed-4c66-b9d9-9f3980289c96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:32.170350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a2a241a-37ed-4c66-b9d9-9f3980289c96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:32.170982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a2a241a-37ed-4c66-b9d9-9f3980289c96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:32.171468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a2a241a-37ed-4c66-b9d9-9f3980289c96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:32.318707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6a2a241a-37ed-4c66-b9d9-9f3980289c96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:32.328798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6a2a241a-37ed-4c66-b9d9-9f3980289c96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.178272 Sep 08 13:58:32.342174 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 367/619] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:32 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:32.380909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b906a7e1-10f6-4374-be15-687f6dab52d3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:32.381960 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b906a7e1-10f6-4374-be15-687f6dab52d3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:32.388950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b906a7e1-10f6-4374-be15-687f6dab52d3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:32.389431 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b906a7e1-10f6-4374-be15-687f6dab52d3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:32.389863 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b906a7e1-10f6-4374-be15-687f6dab52d3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:32.390582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b906a7e1-10f6-4374-be15-687f6dab52d3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:32.391251 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b906a7e1-10f6-4374-be15-687f6dab52d3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:32.393549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b906a7e1-10f6-4374-be15-687f6dab52d3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:32.402546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c1de1654-0af2-4a00-92d9-84f4c789d850 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:32.403264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c1de1654-0af2-4a00-92d9-84f4c789d850 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:32.413385 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1de1654-0af2-4a00-92d9-84f4c789d850 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:32.413907 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1de1654-0af2-4a00-92d9-84f4c789d850 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:32.414369 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1de1654-0af2-4a00-92d9-84f4c789d850 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:32.415131 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1de1654-0af2-4a00-92d9-84f4c789d850 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:32.415581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1de1654-0af2-4a00-92d9-84f4c789d850 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:32.416000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1de1654-0af2-4a00-92d9-84f4c789d850 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:32.421836 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-b906a7e1-10f6-4374-be15-687f6dab52d3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 could not be found. Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b906a7e1-10f6-4374-be15-687f6dab52d3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 HTTP/1.1 Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 13:58:32.423819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 13:58:32.425343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:58:32.425343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 13:58:32.425343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 13:58:32.425343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 13:58:32.425343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 13:58:32.425343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 13:58:32.425343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 13:58:32.425343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 13:58:32.425343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 13:58:32.425343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 13:58:32.425343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 could not be found. Sep 08 13:58:32.425343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '82890cea-b89a-4020-b9aa-8a7d94ac6005'} Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 could not be found. Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:58:32.426311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:58:32.427701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:58:32.427701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 13:58:32.427701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 13:58:32.427701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 13:58:32.427701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 13:58:32.427701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 13:58:32.427701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 13:58:32.427701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 could not be found. Sep 08 13:58:32.427701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:58:32.427701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:58:32.427701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:58:32.427701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:58:32.427701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:58:32.427701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:58:32.427701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:58:32.427701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:58:32.427701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 could not be found. Sep 08 13:58:32.427701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:58:32.428890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b906a7e1-10f6-4374-be15-687f6dab52d3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:58:32.428890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b906a7e1-10f6-4374-be15-687f6dab52d3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 404 len: 111 microversion: 2.1 time: 0.049273 Sep 08 13:58:32.428890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 253/620] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:32 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 13:58:32.434936 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3364f402-389c-40e0-8ea5-c54adaafa9b3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '.version_select at 0x7fce0221bc10>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:32.492821 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3364f402-389c-40e0-8ea5-c54adaafa9b3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-52760673" status: 202 len: 0 microversion: 2.1 time: 0.062448 Sep 08 13:58:32.493703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 254/621] 10.222.0.25 () {58 vars in 1215 bytes} [Thu Sep 8 13:58:32 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-52760673 => generated 0 bytes in 64 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 08 13:58:32.564484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c1de1654-0af2-4a00-92d9-84f4c789d850 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:32.572532 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c1de1654-0af2-4a00-92d9-84f4c789d850 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1272 microversion: 2.1 time: 0.179419 Sep 08 13:58:32.573120 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 368/622] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:32 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1272 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:32.952534 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-99654320-bb8a-4c99-ad5a-e015383ebc00 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:32.953235 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-99654320-bb8a-4c99-ad5a-e015383ebc00 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:32.959157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-99654320-bb8a-4c99-ad5a-e015383ebc00 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:32.959668 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-99654320-bb8a-4c99-ad5a-e015383ebc00 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:32.960088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-99654320-bb8a-4c99-ad5a-e015383ebc00 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:32.960799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-99654320-bb8a-4c99-ad5a-e015383ebc00 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:32.961254 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-99654320-bb8a-4c99-ad5a-e015383ebc00 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:32.961872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-99654320-bb8a-4c99-ad5a-e015383ebc00 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:33.097768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-99654320-bb8a-4c99-ad5a-e015383ebc00 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:33.106780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-99654320-bb8a-4c99-ad5a-e015383ebc00 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.159548 Sep 08 13:58:33.107364 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 255/623] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:32 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:33.357640 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7402ba35-55a4-47ad-874b-98076ccd2fb3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:33.358306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7402ba35-55a4-47ad-874b-98076ccd2fb3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:33.365180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7402ba35-55a4-47ad-874b-98076ccd2fb3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:33.365682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7402ba35-55a4-47ad-874b-98076ccd2fb3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:33.366121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7402ba35-55a4-47ad-874b-98076ccd2fb3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:33.367102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7402ba35-55a4-47ad-874b-98076ccd2fb3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:33.367594 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7402ba35-55a4-47ad-874b-98076ccd2fb3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:33.368008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7402ba35-55a4-47ad-874b-98076ccd2fb3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:33.449618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f2500d46-0b6e-4584-9df4-0116d76ef961 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:33.450273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f2500d46-0b6e-4584-9df4-0116d76ef961 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:33.457465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f2500d46-0b6e-4584-9df4-0116d76ef961 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:33.457962 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f2500d46-0b6e-4584-9df4-0116d76ef961 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:33.458433 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f2500d46-0b6e-4584-9df4-0116d76ef961 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:33.459172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f2500d46-0b6e-4584-9df4-0116d76ef961 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:33.459620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f2500d46-0b6e-4584-9df4-0116d76ef961 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:33.460048 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f2500d46-0b6e-4584-9df4-0116d76ef961 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:33.530131 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7402ba35-55a4-47ad-874b-98076ccd2fb3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:33.539446 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7402ba35-55a4-47ad-874b-98076ccd2fb3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.186275 Sep 08 13:58:33.540044 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 369/624] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:33 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:33.589839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-f2500d46-0b6e-4584-9df4-0116d76ef961 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lazy-loading 'services' on Instance uuid 483c7eb7-3827-403f-84f3-c7c65dcbe574 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 13:58:33.596978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d32ff0b7-3993-44a1-a8b0-9fedc6df8068 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:33.597722 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d32ff0b7-3993-44a1-a8b0-9fedc6df8068 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:33.604583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d32ff0b7-3993-44a1-a8b0-9fedc6df8068 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:33.605157 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d32ff0b7-3993-44a1-a8b0-9fedc6df8068 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:33.605603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d32ff0b7-3993-44a1-a8b0-9fedc6df8068 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:33.606926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d32ff0b7-3993-44a1-a8b0-9fedc6df8068 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:33.607418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d32ff0b7-3993-44a1-a8b0-9fedc6df8068 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:33.607847 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d32ff0b7-3993-44a1-a8b0-9fedc6df8068 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:33.755282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d32ff0b7-3993-44a1-a8b0-9fedc6df8068 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:33.763951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d32ff0b7-3993-44a1-a8b0-9fedc6df8068 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1272 microversion: 2.1 time: 0.171273 Sep 08 13:58:33.764510 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 370/625] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:33 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1272 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:33.879662 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f2500d46-0b6e-4584-9df4-0116d76ef961 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments/06ab5239-fc50-4a85-aa7f-2e9dbf1b3e9d" status: 202 len: 0 microversion: 2.1 time: 0.433865 Sep 08 13:58:33.880261 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 256/626] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 13:58:33 2022] DELETE /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments/06ab5239-fc50-4a85-aa7f-2e9dbf1b3e9d => generated 0 bytes in 435 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 08 13:58:33.888769 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bf97fe3c-9370-4f3e-a10f-2bdc7356b5d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:33.890949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bf97fe3c-9370-4f3e-a10f-2bdc7356b5d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:33.896892 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bf97fe3c-9370-4f3e-a10f-2bdc7356b5d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:33.897384 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bf97fe3c-9370-4f3e-a10f-2bdc7356b5d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:33.897822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bf97fe3c-9370-4f3e-a10f-2bdc7356b5d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:33.898541 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bf97fe3c-9370-4f3e-a10f-2bdc7356b5d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:33.899001 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bf97fe3c-9370-4f3e-a10f-2bdc7356b5d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:33.899428 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bf97fe3c-9370-4f3e-a10f-2bdc7356b5d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:33.930119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bf97fe3c-9370-4f3e-a10f-2bdc7356b5d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045315 Sep 08 13:58:33.930701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 371/627] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 13:58:33 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:58:34.130664 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d66f14bd-00cd-4751-88f9-2cf42acce239 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:34.131399 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d66f14bd-00cd-4751-88f9-2cf42acce239 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:34.137791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d66f14bd-00cd-4751-88f9-2cf42acce239 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:34.138307 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d66f14bd-00cd-4751-88f9-2cf42acce239 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:34.138745 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d66f14bd-00cd-4751-88f9-2cf42acce239 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:34.139437 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d66f14bd-00cd-4751-88f9-2cf42acce239 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:34.139888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d66f14bd-00cd-4751-88f9-2cf42acce239 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:34.140307 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d66f14bd-00cd-4751-88f9-2cf42acce239 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:34.275164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d66f14bd-00cd-4751-88f9-2cf42acce239 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:34.284060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d66f14bd-00cd-4751-88f9-2cf42acce239 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1358 microversion: 2.1 time: 0.157733 Sep 08 13:58:34.284627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 257/628] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:34 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1358 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:34.563609 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e9f1518f-dd2a-4f52-9e11-a8311dcb41b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:34.564232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e9f1518f-dd2a-4f52-9e11-a8311dcb41b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:34.571612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e9f1518f-dd2a-4f52-9e11-a8311dcb41b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:34.577523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e9f1518f-dd2a-4f52-9e11-a8311dcb41b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:34.577523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e9f1518f-dd2a-4f52-9e11-a8311dcb41b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:34.577523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e9f1518f-dd2a-4f52-9e11-a8311dcb41b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:34.577523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e9f1518f-dd2a-4f52-9e11-a8311dcb41b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:34.577989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e9f1518f-dd2a-4f52-9e11-a8311dcb41b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:34.710185 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e9f1518f-dd2a-4f52-9e11-a8311dcb41b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:34.719097 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e9f1518f-dd2a-4f52-9e11-a8311dcb41b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.159632 Sep 08 13:58:34.719716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 372/629] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:34 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:34.788301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-47cce50d-b20f-49dc-9ce9-35bfe7f7a587 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:34.789015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-47cce50d-b20f-49dc-9ce9-35bfe7f7a587 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:34.795594 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-47cce50d-b20f-49dc-9ce9-35bfe7f7a587 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:34.796081 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-47cce50d-b20f-49dc-9ce9-35bfe7f7a587 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:34.796509 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-47cce50d-b20f-49dc-9ce9-35bfe7f7a587 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:34.797204 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-47cce50d-b20f-49dc-9ce9-35bfe7f7a587 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:34.797652 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-47cce50d-b20f-49dc-9ce9-35bfe7f7a587 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:34.798069 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-47cce50d-b20f-49dc-9ce9-35bfe7f7a587 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:34.943837 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-03fd64fd-b6a9-4366-b756-0794a6029d90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:34.946332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-03fd64fd-b6a9-4366-b756-0794a6029d90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:34.953241 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-03fd64fd-b6a9-4366-b756-0794a6029d90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:34.953842 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-03fd64fd-b6a9-4366-b756-0794a6029d90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:34.954624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-03fd64fd-b6a9-4366-b756-0794a6029d90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:34.955437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-03fd64fd-b6a9-4366-b756-0794a6029d90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:34.959439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-03fd64fd-b6a9-4366-b756-0794a6029d90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:34.959439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-03fd64fd-b6a9-4366-b756-0794a6029d90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:34.986868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-47cce50d-b20f-49dc-9ce9-35bfe7f7a587 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:34.992979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:34.998612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-47cce50d-b20f-49dc-9ce9-35bfe7f7a587 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1272 microversion: 2.1 time: 0.214072 Sep 08 13:58:34.998916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-03fd64fd-b6a9-4366-b756-0794a6029d90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059053 Sep 08 13:58:34.999857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 373/630] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 13:58:34 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:58:35.002514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 258/631] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:34 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1272 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:35.307669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-08abc129-08ae-42c2-9cec-624233220a92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:35.308198 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-08abc129-08ae-42c2-9cec-624233220a92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:35.314964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08abc129-08ae-42c2-9cec-624233220a92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:35.315457 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08abc129-08ae-42c2-9cec-624233220a92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:35.315895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08abc129-08ae-42c2-9cec-624233220a92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:35.316601 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08abc129-08ae-42c2-9cec-624233220a92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:35.317053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08abc129-08ae-42c2-9cec-624233220a92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:35.317483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08abc129-08ae-42c2-9cec-624233220a92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:35.447544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-08abc129-08ae-42c2-9cec-624233220a92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:35.455793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-08abc129-08ae-42c2-9cec-624233220a92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1546 microversion: 2.1 time: 0.152440 Sep 08 13:58:35.456290 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 374/632] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:35 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1546 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:35.478641 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6a1aae5e-438c-4ece-bf97-f28e69f5700a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:35.479245 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6a1aae5e-438c-4ece-bf97-f28e69f5700a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:35.485917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a1aae5e-438c-4ece-bf97-f28e69f5700a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:35.486438 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a1aae5e-438c-4ece-bf97-f28e69f5700a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:35.486866 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a1aae5e-438c-4ece-bf97-f28e69f5700a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:35.487557 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a1aae5e-438c-4ece-bf97-f28e69f5700a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:35.488004 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a1aae5e-438c-4ece-bf97-f28e69f5700a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:35.488420 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a1aae5e-438c-4ece-bf97-f28e69f5700a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:35.630163 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6a1aae5e-438c-4ece-bf97-f28e69f5700a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:35.638572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6a1aae5e-438c-4ece-bf97-f28e69f5700a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1546 microversion: 2.1 time: 0.163620 Sep 08 13:58:35.639159 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 259/633] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:35 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1546 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:35.660619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0916cd3a-dc4b-4ad2-ad2f-83fe4da83139 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:35.665535 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0916cd3a-dc4b-4ad2-ad2f-83fe4da83139 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:35.666217 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0916cd3a-dc4b-4ad2-ad2f-83fe4da83139 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Searching by: {'deleted': False, 'project_id': '179292c42f8e4041adc935c80dba0f70'} {{(pid=127601) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 08 13:58:35.673991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0916cd3a-dc4b-4ad2-ad2f-83fe4da83139 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:35.674502 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0916cd3a-dc4b-4ad2-ad2f-83fe4da83139 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:35.674939 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0916cd3a-dc4b-4ad2-ad2f-83fe4da83139 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:35.675688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0916cd3a-dc4b-4ad2-ad2f-83fe4da83139 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:35.676142 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0916cd3a-dc4b-4ad2-ad2f-83fe4da83139 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:35.676572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0916cd3a-dc4b-4ad2-ad2f-83fe4da83139 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:35.742260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3b6fc0a8-d15f-40df-94b0-04cce32eee4c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:35.742865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3b6fc0a8-d15f-40df-94b0-04cce32eee4c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:35.749075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b6fc0a8-d15f-40df-94b0-04cce32eee4c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:35.749672 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b6fc0a8-d15f-40df-94b0-04cce32eee4c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:35.749992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b6fc0a8-d15f-40df-94b0-04cce32eee4c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:35.750720 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b6fc0a8-d15f-40df-94b0-04cce32eee4c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:35.751179 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b6fc0a8-d15f-40df-94b0-04cce32eee4c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:35.751604 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b6fc0a8-d15f-40df-94b0-04cce32eee4c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:35.768138 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.multi_cell_list [None req-0916cd3a-dc4b-4ad2-ad2f-83fe4da83139 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=127601) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 08 13:58:35.867743 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0916cd3a-dc4b-4ad2-ad2f-83fe4da83139 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.211191 Sep 08 13:58:35.868637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 375/634] 10.222.0.25 () {58 vars in 1125 bytes} [Thu Sep 8 13:58:35 2022] GET /compute/v2.1/servers => generated 343 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:58:35.880220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1496b4bd-bc60-42de-9602-03acb6c51820 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:35.880827 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1496b4bd-bc60-42de-9602-03acb6c51820 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:35.887015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3b6fc0a8-d15f-40df-94b0-04cce32eee4c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:35.887867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1496b4bd-bc60-42de-9602-03acb6c51820 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:35.887867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1496b4bd-bc60-42de-9602-03acb6c51820 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:35.888090 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1496b4bd-bc60-42de-9602-03acb6c51820 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:35.888644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1496b4bd-bc60-42de-9602-03acb6c51820 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:35.889098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1496b4bd-bc60-42de-9602-03acb6c51820 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:35.889528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1496b4bd-bc60-42de-9602-03acb6c51820 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:35.896782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3b6fc0a8-d15f-40df-94b0-04cce32eee4c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.158430 Sep 08 13:58:35.897476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 260/635] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:35 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:36.012020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7f87ebae-5dbe-4041-9874-c861d7f84e84 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:36.015168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7f87ebae-5dbe-4041-9874-c861d7f84e84 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:36.021649 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7f87ebae-5dbe-4041-9874-c861d7f84e84 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:36.022585 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7f87ebae-5dbe-4041-9874-c861d7f84e84 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:36.023028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7f87ebae-5dbe-4041-9874-c861d7f84e84 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:36.023740 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7f87ebae-5dbe-4041-9874-c861d7f84e84 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:36.024197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7f87ebae-5dbe-4041-9874-c861d7f84e84 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:36.024444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1496b4bd-bc60-42de-9602-03acb6c51820 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:36.024621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7f87ebae-5dbe-4041-9874-c861d7f84e84 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:36.037251 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1496b4bd-bc60-42de-9602-03acb6c51820 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1546 microversion: 2.1 time: 0.160780 Sep 08 13:58:36.038031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 376/636] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:35 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1546 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:36.042518 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-00bb4430-b06c-420d-8d15-ceea236b24bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:36.043126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-00bb4430-b06c-420d-8d15-ceea236b24bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:36.049429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-00bb4430-b06c-420d-8d15-ceea236b24bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:36.049929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-00bb4430-b06c-420d-8d15-ceea236b24bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:36.050458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-00bb4430-b06c-420d-8d15-ceea236b24bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:36.051171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-00bb4430-b06c-420d-8d15-ceea236b24bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:36.051840 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-00bb4430-b06c-420d-8d15-ceea236b24bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:36.052117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-00bb4430-b06c-420d-8d15-ceea236b24bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:36.071168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7f87ebae-5dbe-4041-9874-c861d7f84e84 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063282 Sep 08 13:58:36.071756 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 261/637] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 13:58:36 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:58:36.231263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-00bb4430-b06c-420d-8d15-ceea236b24bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:36.249456 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-00bb4430-b06c-420d-8d15-ceea236b24bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1272 microversion: 2.1 time: 0.210191 Sep 08 13:58:36.258301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 377/638] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:36 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1272 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:36.923903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4987acdd-6e30-44f8-8a28-8cd8e6b6a48c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:36.926902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4987acdd-6e30-44f8-8a28-8cd8e6b6a48c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:36.932541 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4987acdd-6e30-44f8-8a28-8cd8e6b6a48c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:36.933036 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4987acdd-6e30-44f8-8a28-8cd8e6b6a48c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:36.933470 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4987acdd-6e30-44f8-8a28-8cd8e6b6a48c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:36.934263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4987acdd-6e30-44f8-8a28-8cd8e6b6a48c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:36.935169 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4987acdd-6e30-44f8-8a28-8cd8e6b6a48c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:36.935829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4987acdd-6e30-44f8-8a28-8cd8e6b6a48c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:37.084475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-364c981b-248a-4569-be86-645e1edb1c97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:37.086649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-364c981b-248a-4569-be86-645e1edb1c97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:37.094836 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-364c981b-248a-4569-be86-645e1edb1c97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:37.095358 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-364c981b-248a-4569-be86-645e1edb1c97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:37.095947 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-364c981b-248a-4569-be86-645e1edb1c97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:37.096646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-364c981b-248a-4569-be86-645e1edb1c97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:37.097088 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-364c981b-248a-4569-be86-645e1edb1c97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:37.097507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-364c981b-248a-4569-be86-645e1edb1c97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:37.130829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-364c981b-248a-4569-be86-645e1edb1c97 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050262 Sep 08 13:58:37.132125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 378/639] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 13:58:37 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:58:37.165413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4987acdd-6e30-44f8-8a28-8cd8e6b6a48c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:37.174359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4987acdd-6e30-44f8-8a28-8cd8e6b6a48c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.257037 Sep 08 13:58:37.175026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 262/640] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:36 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:37.273591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c3ee1fea-15e2-49d5-9608-64c4d457d891 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:37.274479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c3ee1fea-15e2-49d5-9608-64c4d457d891 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:37.281259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3ee1fea-15e2-49d5-9608-64c4d457d891 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:37.282020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3ee1fea-15e2-49d5-9608-64c4d457d891 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:37.282489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3ee1fea-15e2-49d5-9608-64c4d457d891 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:37.283311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3ee1fea-15e2-49d5-9608-64c4d457d891 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:37.283770 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3ee1fea-15e2-49d5-9608-64c4d457d891 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:37.284202 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3ee1fea-15e2-49d5-9608-64c4d457d891 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:37.455047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c3ee1fea-15e2-49d5-9608-64c4d457d891 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:37.466471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c3ee1fea-15e2-49d5-9608-64c4d457d891 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1272 microversion: 2.1 time: 0.197356 Sep 08 13:58:37.467366 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 379/641] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:37 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1272 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:38.143525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ae2a430f-d2fe-40d8-a210-617057fdb808 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:38.145673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ae2a430f-d2fe-40d8-a210-617057fdb808 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:38.152249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ae2a430f-d2fe-40d8-a210-617057fdb808 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:38.152729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ae2a430f-d2fe-40d8-a210-617057fdb808 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:38.153785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ae2a430f-d2fe-40d8-a210-617057fdb808 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:38.154428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ae2a430f-d2fe-40d8-a210-617057fdb808 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:38.154860 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ae2a430f-d2fe-40d8-a210-617057fdb808 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:38.155301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ae2a430f-d2fe-40d8-a210-617057fdb808 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:38.186063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ae2a430f-d2fe-40d8-a210-617057fdb808 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046407 Sep 08 13:58:38.186657 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 263/642] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 13:58:38 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:58:38.199334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4f2b4104-9f76-4834-abbc-822e923c2836 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:38.199334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4f2b4104-9f76-4834-abbc-822e923c2836 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:38.206370 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f2b4104-9f76-4834-abbc-822e923c2836 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:38.206858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f2b4104-9f76-4834-abbc-822e923c2836 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:38.207293 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f2b4104-9f76-4834-abbc-822e923c2836 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:38.207982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f2b4104-9f76-4834-abbc-822e923c2836 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:38.208436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f2b4104-9f76-4834-abbc-822e923c2836 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:38.208859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f2b4104-9f76-4834-abbc-822e923c2836 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:38.347473 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4f2b4104-9f76-4834-abbc-822e923c2836 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:38.356042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4f2b4104-9f76-4834-abbc-822e923c2836 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.161141 Sep 08 13:58:38.356647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 380/643] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:38 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:38.502654 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7a9e0209-5e3f-4d9d-93a6-b1703cb7ea67 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:38.503279 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7a9e0209-5e3f-4d9d-93a6-b1703cb7ea67 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:38.509633 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a9e0209-5e3f-4d9d-93a6-b1703cb7ea67 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:38.510128 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a9e0209-5e3f-4d9d-93a6-b1703cb7ea67 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:38.510593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a9e0209-5e3f-4d9d-93a6-b1703cb7ea67 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:38.511291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a9e0209-5e3f-4d9d-93a6-b1703cb7ea67 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:38.511744 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a9e0209-5e3f-4d9d-93a6-b1703cb7ea67 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:38.512168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a9e0209-5e3f-4d9d-93a6-b1703cb7ea67 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:38.666387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7a9e0209-5e3f-4d9d-93a6-b1703cb7ea67 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:38.674421 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7a9e0209-5e3f-4d9d-93a6-b1703cb7ea67 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1272 microversion: 2.1 time: 0.175682 Sep 08 13:58:38.675016 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 264/644] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:38 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1272 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:39.198359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-629024c3-9721-4211-b3bc-16e9bf893953 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:39.200540 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-629024c3-9721-4211-b3bc-16e9bf893953 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:39.207148 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-629024c3-9721-4211-b3bc-16e9bf893953 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:39.207632 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-629024c3-9721-4211-b3bc-16e9bf893953 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:39.208060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-629024c3-9721-4211-b3bc-16e9bf893953 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:39.208747 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-629024c3-9721-4211-b3bc-16e9bf893953 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:39.209190 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-629024c3-9721-4211-b3bc-16e9bf893953 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:39.209606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-629024c3-9721-4211-b3bc-16e9bf893953 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:39.240682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-629024c3-9721-4211-b3bc-16e9bf893953 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046250 Sep 08 13:58:39.241240 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 381/645] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 13:58:39 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:58:39.279087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f909c7e5-abe7-4944-9602-90da5a89fa0c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9a4342d7-3883-4385-bcd2-b4070743be65"}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:58:39.281435 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f909c7e5-abe7-4944-9602-90da5a89fa0c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:39.288280 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f909c7e5-abe7-4944-9602-90da5a89fa0c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:39.288773 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f909c7e5-abe7-4944-9602-90da5a89fa0c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:39.289207 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f909c7e5-abe7-4944-9602-90da5a89fa0c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:39.289910 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f909c7e5-abe7-4944-9602-90da5a89fa0c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:39.290392 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f909c7e5-abe7-4944-9602-90da5a89fa0c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:39.290817 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f909c7e5-abe7-4944-9602-90da5a89fa0c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:39.381968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a3458fdb-6f28-46b5-b111-4cfd7e660846 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:39.382607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a3458fdb-6f28-46b5-b111-4cfd7e660846 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:39.388872 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a3458fdb-6f28-46b5-b111-4cfd7e660846 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:39.389429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a3458fdb-6f28-46b5-b111-4cfd7e660846 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:39.389867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a3458fdb-6f28-46b5-b111-4cfd7e660846 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:39.390605 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a3458fdb-6f28-46b5-b111-4cfd7e660846 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:39.391058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a3458fdb-6f28-46b5-b111-4cfd7e660846 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:39.391509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a3458fdb-6f28-46b5-b111-4cfd7e660846 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:39.522915 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a3458fdb-6f28-46b5-b111-4cfd7e660846 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:39.531981 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a3458fdb-6f28-46b5-b111-4cfd7e660846 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.154237 Sep 08 13:58:39.532753 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 382/646] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:39 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:39.698008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a370f4c0-2fa6-44a1-bf05-a62db0f65bee tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:39.698849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a370f4c0-2fa6-44a1-bf05-a62db0f65bee tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:39.705300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a370f4c0-2fa6-44a1-bf05-a62db0f65bee tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:39.705787 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a370f4c0-2fa6-44a1-bf05-a62db0f65bee tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:39.706251 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a370f4c0-2fa6-44a1-bf05-a62db0f65bee tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:39.707128 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a370f4c0-2fa6-44a1-bf05-a62db0f65bee tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:39.707825 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a370f4c0-2fa6-44a1-bf05-a62db0f65bee tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:39.708478 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a370f4c0-2fa6-44a1-bf05-a62db0f65bee tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:39.844537 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a370f4c0-2fa6-44a1-bf05-a62db0f65bee tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:39.852313 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a370f4c0-2fa6-44a1-bf05-a62db0f65bee tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1272 microversion: 2.1 time: 0.158340 Sep 08 13:58:39.852872 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 383/647] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:39 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1272 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:40.252879 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-022286ff-7003-45c9-8b56-3e137565d30a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:40.255153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-022286ff-7003-45c9-8b56-3e137565d30a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:40.261442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-022286ff-7003-45c9-8b56-3e137565d30a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:40.261932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-022286ff-7003-45c9-8b56-3e137565d30a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:40.262417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-022286ff-7003-45c9-8b56-3e137565d30a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:40.263108 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-022286ff-7003-45c9-8b56-3e137565d30a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:40.263566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-022286ff-7003-45c9-8b56-3e137565d30a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:40.263991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-022286ff-7003-45c9-8b56-3e137565d30a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:40.294583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-022286ff-7003-45c9-8b56-3e137565d30a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045565 Sep 08 13:58:40.295136 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 384/648] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 13:58:40 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:58:40.556379 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f3c21203-83ff-4e0d-aed6-5c8e80ba90cd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:40.556908 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f3c21203-83ff-4e0d-aed6-5c8e80ba90cd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:40.563146 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f3c21203-83ff-4e0d-aed6-5c8e80ba90cd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:40.563637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f3c21203-83ff-4e0d-aed6-5c8e80ba90cd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:40.564129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f3c21203-83ff-4e0d-aed6-5c8e80ba90cd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:40.564832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f3c21203-83ff-4e0d-aed6-5c8e80ba90cd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:40.565326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f3c21203-83ff-4e0d-aed6-5c8e80ba90cd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:40.565754 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f3c21203-83ff-4e0d-aed6-5c8e80ba90cd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:40.695494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f3c21203-83ff-4e0d-aed6-5c8e80ba90cd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:40.704398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f3c21203-83ff-4e0d-aed6-5c8e80ba90cd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.152001 Sep 08 13:58:40.704958 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 385/649] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:40 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:40.875583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-453e1c5d-a347-4d97-b504-e18b49c9eef8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:40.876187 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-453e1c5d-a347-4d97-b504-e18b49c9eef8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:40.882106 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-453e1c5d-a347-4d97-b504-e18b49c9eef8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:40.882619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-453e1c5d-a347-4d97-b504-e18b49c9eef8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:40.883053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-453e1c5d-a347-4d97-b504-e18b49c9eef8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:40.883823 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-453e1c5d-a347-4d97-b504-e18b49c9eef8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:40.884276 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-453e1c5d-a347-4d97-b504-e18b49c9eef8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:40.884697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-453e1c5d-a347-4d97-b504-e18b49c9eef8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:41.018163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-453e1c5d-a347-4d97-b504-e18b49c9eef8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:41.025859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-453e1c5d-a347-4d97-b504-e18b49c9eef8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1272 microversion: 2.1 time: 0.154226 Sep 08 13:58:41.026454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 386/650] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:40 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1272 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:41.307959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-367c91b7-87b1-46fb-9f38-8b4edd9f21be tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:41.310558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-367c91b7-87b1-46fb-9f38-8b4edd9f21be tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:41.316844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-367c91b7-87b1-46fb-9f38-8b4edd9f21be tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:41.317333 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-367c91b7-87b1-46fb-9f38-8b4edd9f21be tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:41.317887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-367c91b7-87b1-46fb-9f38-8b4edd9f21be tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:41.318691 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-367c91b7-87b1-46fb-9f38-8b4edd9f21be tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:41.319147 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-367c91b7-87b1-46fb-9f38-8b4edd9f21be tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:41.319587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-367c91b7-87b1-46fb-9f38-8b4edd9f21be tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:41.353010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-367c91b7-87b1-46fb-9f38-8b4edd9f21be tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048845 Sep 08 13:58:41.355295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 387/651] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 13:58:41 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:58:41.728344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-38f3c7c4-1fee-4d02-b047-91b611620157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:41.728954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-38f3c7c4-1fee-4d02-b047-91b611620157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:41.734967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38f3c7c4-1fee-4d02-b047-91b611620157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:41.735455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38f3c7c4-1fee-4d02-b047-91b611620157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:41.735903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38f3c7c4-1fee-4d02-b047-91b611620157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:41.736629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38f3c7c4-1fee-4d02-b047-91b611620157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:41.737095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38f3c7c4-1fee-4d02-b047-91b611620157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:41.737526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38f3c7c4-1fee-4d02-b047-91b611620157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:41.867285 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-38f3c7c4-1fee-4d02-b047-91b611620157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:41.876296 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-38f3c7c4-1fee-4d02-b047-91b611620157 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.151911 Sep 08 13:58:41.877445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 388/652] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:41 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:42.049367 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e19f7415-da52-4eb8-a058-219aa2687d0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:42.049969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e19f7415-da52-4eb8-a058-219aa2687d0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:42.056025 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e19f7415-da52-4eb8-a058-219aa2687d0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:42.056511 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e19f7415-da52-4eb8-a058-219aa2687d0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:42.056946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e19f7415-da52-4eb8-a058-219aa2687d0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:42.057631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e19f7415-da52-4eb8-a058-219aa2687d0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:42.058086 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e19f7415-da52-4eb8-a058-219aa2687d0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:42.058547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e19f7415-da52-4eb8-a058-219aa2687d0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:42.242109 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f909c7e5-abe7-4944-9602-90da5a89fa0c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "POST /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 2.966832 Sep 08 13:58:42.243131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 265/653] 10.222.0.25 () {60 vars in 1323 bytes} [Thu Sep 8 13:58:39 2022] POST /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 194 bytes in 2969 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:58:42.366629 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-aa6377fb-c4c7-4f76-b041-5d15eca7533d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:42.369297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-aa6377fb-c4c7-4f76-b041-5d15eca7533d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:42.375818 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa6377fb-c4c7-4f76-b041-5d15eca7533d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:42.377160 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa6377fb-c4c7-4f76-b041-5d15eca7533d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:42.377160 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa6377fb-c4c7-4f76-b041-5d15eca7533d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:42.377616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa6377fb-c4c7-4f76-b041-5d15eca7533d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:42.378006 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa6377fb-c4c7-4f76-b041-5d15eca7533d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:42.378460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa6377fb-c4c7-4f76-b041-5d15eca7533d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:42.396567 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=96,nova_cell1:INSERT=2,nova_cell1:UPDATE=2 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:42.411829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-aa6377fb-c4c7-4f76-b041-5d15eca7533d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049727 Sep 08 13:58:42.412408 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 266/654] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 13:58:42 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:58:42.414586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e19f7415-da52-4eb8-a058-219aa2687d0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:42.424054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e19f7415-da52-4eb8-a058-219aa2687d0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1272 microversion: 2.1 time: 0.379021 Sep 08 13:58:42.424623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 389/655] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:42 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1272 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:42.900561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-782963a2-f816-4dc5-a240-069519a5a698 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:42.901210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-782963a2-f816-4dc5-a240-069519a5a698 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:42.912546 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-782963a2-f816-4dc5-a240-069519a5a698 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:42.912546 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-782963a2-f816-4dc5-a240-069519a5a698 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:42.912546 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-782963a2-f816-4dc5-a240-069519a5a698 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:42.912546 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-782963a2-f816-4dc5-a240-069519a5a698 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:42.913049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-782963a2-f816-4dc5-a240-069519a5a698 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:42.913049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-782963a2-f816-4dc5-a240-069519a5a698 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:43.047956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-782963a2-f816-4dc5-a240-069519a5a698 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:43.057163 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-782963a2-f816-4dc5-a240-069519a5a698 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.160894 Sep 08 13:58:43.057772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 267/656] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:42 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:43.425364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-030189a6-5574-4c0c-9eb7-41f01fbb3048 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:43.427623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-030189a6-5574-4c0c-9eb7-41f01fbb3048 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:43.434199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-030189a6-5574-4c0c-9eb7-41f01fbb3048 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:43.434739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-030189a6-5574-4c0c-9eb7-41f01fbb3048 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:43.435253 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-030189a6-5574-4c0c-9eb7-41f01fbb3048 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:43.435958 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-030189a6-5574-4c0c-9eb7-41f01fbb3048 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:43.436417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-030189a6-5574-4c0c-9eb7-41f01fbb3048 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:43.436841 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-030189a6-5574-4c0c-9eb7-41f01fbb3048 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:43.439574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:43.446472 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f589b6d4-e2e1-4e89-a654-9272f33b61b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:43.447087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f589b6d4-e2e1-4e89-a654-9272f33b61b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:43.454288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f589b6d4-e2e1-4e89-a654-9272f33b61b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:43.454853 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f589b6d4-e2e1-4e89-a654-9272f33b61b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:43.455217 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f589b6d4-e2e1-4e89-a654-9272f33b61b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:43.456202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f589b6d4-e2e1-4e89-a654-9272f33b61b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:43.456706 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f589b6d4-e2e1-4e89-a654-9272f33b61b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:43.457133 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f589b6d4-e2e1-4e89-a654-9272f33b61b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:43.473617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-030189a6-5574-4c0c-9eb7-41f01fbb3048 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052848 Sep 08 13:58:43.475315 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 390/657] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 13:58:43 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:58:43.598391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f589b6d4-e2e1-4e89-a654-9272f33b61b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:43.606076 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f589b6d4-e2e1-4e89-a654-9272f33b61b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.163924 Sep 08 13:58:43.607122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 268/658] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:43 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:44.081467 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e91290dd-7140-4e30-a4a4-09ad38e046ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:44.081990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e91290dd-7140-4e30-a4a4-09ad38e046ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:44.088579 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e91290dd-7140-4e30-a4a4-09ad38e046ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:44.089063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e91290dd-7140-4e30-a4a4-09ad38e046ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:44.089499 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e91290dd-7140-4e30-a4a4-09ad38e046ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:44.090220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e91290dd-7140-4e30-a4a4-09ad38e046ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:44.090678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e91290dd-7140-4e30-a4a4-09ad38e046ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:44.091105 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e91290dd-7140-4e30-a4a4-09ad38e046ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:44.226024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e91290dd-7140-4e30-a4a4-09ad38e046ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:44.235002 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e91290dd-7140-4e30-a4a4-09ad38e046ef tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.158035 Sep 08 13:58:44.235562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 391/659] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:44 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:44.593529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1c6420f2-950a-4e13-8d10-c1b524ba72fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:44.595882 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1c6420f2-950a-4e13-8d10-c1b524ba72fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:44.601979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c6420f2-950a-4e13-8d10-c1b524ba72fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:44.602490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c6420f2-950a-4e13-8d10-c1b524ba72fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:44.602929 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c6420f2-950a-4e13-8d10-c1b524ba72fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:44.603649 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c6420f2-950a-4e13-8d10-c1b524ba72fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:44.604103 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c6420f2-950a-4e13-8d10-c1b524ba72fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:44.604520 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c6420f2-950a-4e13-8d10-c1b524ba72fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:44.633657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-12c7fc9c-2028-4b80-90a9-98e0fe0290b1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:44.633657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-12c7fc9c-2028-4b80-90a9-98e0fe0290b1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:44.634834 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1c6420f2-950a-4e13-8d10-c1b524ba72fc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.151640 Sep 08 13:58:44.635582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 269/660] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 13:58:44 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574/os-volume_attachments => generated 25 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 08 13:58:44.636625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-12c7fc9c-2028-4b80-90a9-98e0fe0290b1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:44.637127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-12c7fc9c-2028-4b80-90a9-98e0fe0290b1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:44.637572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-12c7fc9c-2028-4b80-90a9-98e0fe0290b1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:44.638308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-12c7fc9c-2028-4b80-90a9-98e0fe0290b1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:44.638765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-12c7fc9c-2028-4b80-90a9-98e0fe0290b1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:44.639190 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-12c7fc9c-2028-4b80-90a9-98e0fe0290b1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:44.704157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1c87f700-c484-4591-b185-0260a139b9d4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:44.704745 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1c87f700-c484-4591-b185-0260a139b9d4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:44.710772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c87f700-c484-4591-b185-0260a139b9d4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:44.711335 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c87f700-c484-4591-b185-0260a139b9d4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:44.711878 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c87f700-c484-4591-b185-0260a139b9d4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:44.712640 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c87f700-c484-4591-b185-0260a139b9d4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:44.713100 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c87f700-c484-4591-b185-0260a139b9d4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:44.713533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c87f700-c484-4591-b185-0260a139b9d4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:44.739310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1c87f700-c484-4591-b185-0260a139b9d4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 13:58:44.776169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-12c7fc9c-2028-4b80-90a9-98e0fe0290b1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:44.784126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-12c7fc9c-2028-4b80-90a9-98e0fe0290b1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.158629 Sep 08 13:58:44.784701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 392/661] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:44 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:45.054878 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1c87f700-c484-4591-b185-0260a139b9d4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 204 len: 0 microversion: 2.1 time: 0.354294 Sep 08 13:58:45.055411 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 270/662] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 13:58:44 2022] DELETE /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 08 13:58:45.063532 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c1fcb809-c684-4d03-a57e-323458574c9f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:45.068445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c1fcb809-c684-4d03-a57e-323458574c9f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:45.070013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1fcb809-c684-4d03-a57e-323458574c9f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:45.070526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1fcb809-c684-4d03-a57e-323458574c9f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:45.070963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1fcb809-c684-4d03-a57e-323458574c9f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:45.071651 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1fcb809-c684-4d03-a57e-323458574c9f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:45.072092 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1fcb809-c684-4d03-a57e-323458574c9f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:45.072506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1fcb809-c684-4d03-a57e-323458574c9f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:45.205155 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c1fcb809-c684-4d03-a57e-323458574c9f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:45.213576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c1fcb809-c684-4d03-a57e-323458574c9f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.153805 Sep 08 13:58:45.214267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 393/663] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:45 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:45.259606 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d9ad4bf7-8721-439e-ade4-87c596affe5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:45.260301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d9ad4bf7-8721-439e-ade4-87c596affe5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:45.266509 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d9ad4bf7-8721-439e-ade4-87c596affe5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:45.267084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d9ad4bf7-8721-439e-ade4-87c596affe5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:45.267535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d9ad4bf7-8721-439e-ade4-87c596affe5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:45.268230 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d9ad4bf7-8721-439e-ade4-87c596affe5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:45.268686 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d9ad4bf7-8721-439e-ade4-87c596affe5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:45.269116 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d9ad4bf7-8721-439e-ade4-87c596affe5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:45.402789 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d9ad4bf7-8721-439e-ade4-87c596affe5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:45.412445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d9ad4bf7-8721-439e-ade4-87c596affe5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.156672 Sep 08 13:58:45.413457 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 271/664] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:45 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:45.819342 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5066a23c-f812-4048-a8c9-84d927f2e4a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:45.819979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5066a23c-f812-4048-a8c9-84d927f2e4a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:45.822349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=2 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:45.827295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5066a23c-f812-4048-a8c9-84d927f2e4a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:45.827792 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5066a23c-f812-4048-a8c9-84d927f2e4a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:45.828944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5066a23c-f812-4048-a8c9-84d927f2e4a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:45.829727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5066a23c-f812-4048-a8c9-84d927f2e4a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:45.830526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5066a23c-f812-4048-a8c9-84d927f2e4a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:45.831006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5066a23c-f812-4048-a8c9-84d927f2e4a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:45.960068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5066a23c-f812-4048-a8c9-84d927f2e4a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:45.967907 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5066a23c-f812-4048-a8c9-84d927f2e4a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.152444 Sep 08 13:58:45.968463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 394/665] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:45 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:46.237338 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8fea44ee-e7b8-4f8b-8505-29c8fbae6178 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:46.237981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8fea44ee-e7b8-4f8b-8505-29c8fbae6178 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:46.244840 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fea44ee-e7b8-4f8b-8505-29c8fbae6178 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:46.245326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fea44ee-e7b8-4f8b-8505-29c8fbae6178 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:46.245761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fea44ee-e7b8-4f8b-8505-29c8fbae6178 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:46.246485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fea44ee-e7b8-4f8b-8505-29c8fbae6178 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:46.246944 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fea44ee-e7b8-4f8b-8505-29c8fbae6178 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:46.247367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fea44ee-e7b8-4f8b-8505-29c8fbae6178 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:46.377982 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8fea44ee-e7b8-4f8b-8505-29c8fbae6178 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:46.385806 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8fea44ee-e7b8-4f8b-8505-29c8fbae6178 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.152851 Sep 08 13:58:46.386410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 272/666] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:46 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:46.435595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a7e2d710-8899-4376-bf48-7f3f3d5e6b4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:46.436264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a7e2d710-8899-4376-bf48-7f3f3d5e6b4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:46.443201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7e2d710-8899-4376-bf48-7f3f3d5e6b4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:46.443691 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7e2d710-8899-4376-bf48-7f3f3d5e6b4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:46.444127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7e2d710-8899-4376-bf48-7f3f3d5e6b4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:46.444867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7e2d710-8899-4376-bf48-7f3f3d5e6b4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:46.445316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7e2d710-8899-4376-bf48-7f3f3d5e6b4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:46.445738 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7e2d710-8899-4376-bf48-7f3f3d5e6b4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:46.581082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a7e2d710-8899-4376-bf48-7f3f3d5e6b4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:46.589719 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a7e2d710-8899-4376-bf48-7f3f3d5e6b4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.157933 Sep 08 13:58:46.590537 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 395/667] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:46 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:46.989724 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9cebfa36-939e-4cf9-8721-4640c16d12d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:46.990365 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9cebfa36-939e-4cf9-8721-4640c16d12d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:46.996321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9cebfa36-939e-4cf9-8721-4640c16d12d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:46.996797 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9cebfa36-939e-4cf9-8721-4640c16d12d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:46.997225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9cebfa36-939e-4cf9-8721-4640c16d12d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:46.997914 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9cebfa36-939e-4cf9-8721-4640c16d12d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:46.998385 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9cebfa36-939e-4cf9-8721-4640c16d12d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:46.998804 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9cebfa36-939e-4cf9-8721-4640c16d12d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:47.139908 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9cebfa36-939e-4cf9-8721-4640c16d12d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:47.148655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9cebfa36-939e-4cf9-8721-4640c16d12d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.162472 Sep 08 13:58:47.149009 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 273/668] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:46 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:47.413280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-81fe31c9-9238-4864-9628-d0279af68f02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:47.413893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-81fe31c9-9238-4864-9628-d0279af68f02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:47.420474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81fe31c9-9238-4864-9628-d0279af68f02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:47.420963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81fe31c9-9238-4864-9628-d0279af68f02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:47.421401 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81fe31c9-9238-4864-9628-d0279af68f02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:47.422126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81fe31c9-9238-4864-9628-d0279af68f02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:47.422605 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81fe31c9-9238-4864-9628-d0279af68f02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:47.423028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81fe31c9-9238-4864-9628-d0279af68f02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:47.554065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-81fe31c9-9238-4864-9628-d0279af68f02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:47.561325 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-81fe31c9-9238-4864-9628-d0279af68f02 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.151920 Sep 08 13:58:47.561893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 396/669] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:47 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:47.711410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-dd00c2f1-462e-41c6-ba6a-e5584325d5d4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:47.712019 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-dd00c2f1-462e-41c6-ba6a-e5584325d5d4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:47.718320 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd00c2f1-462e-41c6-ba6a-e5584325d5d4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:47.718808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd00c2f1-462e-41c6-ba6a-e5584325d5d4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:47.719244 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd00c2f1-462e-41c6-ba6a-e5584325d5d4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:47.719939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd00c2f1-462e-41c6-ba6a-e5584325d5d4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:47.720402 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd00c2f1-462e-41c6-ba6a-e5584325d5d4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:47.720831 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd00c2f1-462e-41c6-ba6a-e5584325d5d4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:47.853822 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-dd00c2f1-462e-41c6-ba6a-e5584325d5d4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:47.862778 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-dd00c2f1-462e-41c6-ba6a-e5584325d5d4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.253508 Sep 08 13:58:47.863349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 274/670] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:47 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:48.172663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9542e7ce-10ed-4b6f-95a3-0aaba30bb675 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:48.173296 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9542e7ce-10ed-4b6f-95a3-0aaba30bb675 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:48.179553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9542e7ce-10ed-4b6f-95a3-0aaba30bb675 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:48.180042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9542e7ce-10ed-4b6f-95a3-0aaba30bb675 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:48.180478 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9542e7ce-10ed-4b6f-95a3-0aaba30bb675 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:48.181168 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9542e7ce-10ed-4b6f-95a3-0aaba30bb675 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:48.181625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9542e7ce-10ed-4b6f-95a3-0aaba30bb675 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:48.182051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9542e7ce-10ed-4b6f-95a3-0aaba30bb675 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:48.318525 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9542e7ce-10ed-4b6f-95a3-0aaba30bb675 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:48.326455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9542e7ce-10ed-4b6f-95a3-0aaba30bb675 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.158292 Sep 08 13:58:48.327199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 397/671] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:48 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:48.585723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f805f283-430f-46d8-a67b-262a29b42dc3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:48.586357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f805f283-430f-46d8-a67b-262a29b42dc3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:48.592326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f805f283-430f-46d8-a67b-262a29b42dc3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:48.592808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f805f283-430f-46d8-a67b-262a29b42dc3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:48.593244 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f805f283-430f-46d8-a67b-262a29b42dc3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:48.594588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f805f283-430f-46d8-a67b-262a29b42dc3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:48.595129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f805f283-430f-46d8-a67b-262a29b42dc3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:48.595559 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f805f283-430f-46d8-a67b-262a29b42dc3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:48.732599 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f805f283-430f-46d8-a67b-262a29b42dc3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:48.742369 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f805f283-430f-46d8-a67b-262a29b42dc3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.160410 Sep 08 13:58:48.742951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 275/672] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:48 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:48.887445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-eb2cafbe-ea78-4e61-941a-1c3ed6d04529 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:48.887995 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-eb2cafbe-ea78-4e61-941a-1c3ed6d04529 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:48.895080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb2cafbe-ea78-4e61-941a-1c3ed6d04529 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:48.895567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb2cafbe-ea78-4e61-941a-1c3ed6d04529 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:48.896008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb2cafbe-ea78-4e61-941a-1c3ed6d04529 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:48.896705 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb2cafbe-ea78-4e61-941a-1c3ed6d04529 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:48.897164 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb2cafbe-ea78-4e61-941a-1c3ed6d04529 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:48.897595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb2cafbe-ea78-4e61-941a-1c3ed6d04529 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:49.029963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-eb2cafbe-ea78-4e61-941a-1c3ed6d04529 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:49.038556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-eb2cafbe-ea78-4e61-941a-1c3ed6d04529 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.155189 Sep 08 13:58:49.039126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 398/673] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:48 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:49.349654 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-dfa30734-b005-4188-8a51-c77588c315de tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:49.350274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-dfa30734-b005-4188-8a51-c77588c315de tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:49.356248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dfa30734-b005-4188-8a51-c77588c315de tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:49.356723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dfa30734-b005-4188-8a51-c77588c315de tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:49.357149 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dfa30734-b005-4188-8a51-c77588c315de tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:49.357825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dfa30734-b005-4188-8a51-c77588c315de tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:49.358297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dfa30734-b005-4188-8a51-c77588c315de tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:49.358716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dfa30734-b005-4188-8a51-c77588c315de tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:49.490581 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-dfa30734-b005-4188-8a51-c77588c315de tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:49.498368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-dfa30734-b005-4188-8a51-c77588c315de tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.152702 Sep 08 13:58:49.498951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 276/674] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:49 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:49.766617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-828cb660-e1e7-4b3f-857c-3b534d10950a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:49.767377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-828cb660-e1e7-4b3f-857c-3b534d10950a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:49.774800 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-828cb660-e1e7-4b3f-857c-3b534d10950a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:49.775638 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-828cb660-e1e7-4b3f-857c-3b534d10950a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:49.775739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-828cb660-e1e7-4b3f-857c-3b534d10950a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:49.776761 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-828cb660-e1e7-4b3f-857c-3b534d10950a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:49.777301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-828cb660-e1e7-4b3f-857c-3b534d10950a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:49.777723 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-828cb660-e1e7-4b3f-857c-3b534d10950a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:49.918085 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-828cb660-e1e7-4b3f-857c-3b534d10950a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:49.926512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-828cb660-e1e7-4b3f-857c-3b534d10950a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.163647 Sep 08 13:58:49.927071 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 399/675] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:49 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:50.061859 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-31dc2eb6-4ad0-476d-bb18-c5acc15ebd1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:50.063133 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-31dc2eb6-4ad0-476d-bb18-c5acc15ebd1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:50.069795 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31dc2eb6-4ad0-476d-bb18-c5acc15ebd1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:50.070320 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31dc2eb6-4ad0-476d-bb18-c5acc15ebd1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:50.070754 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31dc2eb6-4ad0-476d-bb18-c5acc15ebd1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:50.071439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31dc2eb6-4ad0-476d-bb18-c5acc15ebd1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:50.071891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31dc2eb6-4ad0-476d-bb18-c5acc15ebd1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:50.072311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31dc2eb6-4ad0-476d-bb18-c5acc15ebd1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:50.216534 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-31dc2eb6-4ad0-476d-bb18-c5acc15ebd1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:50.225318 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-31dc2eb6-4ad0-476d-bb18-c5acc15ebd1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.167289 Sep 08 13:58:50.225885 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 277/676] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:50 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:50.521068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f2eabeb5-695e-47ff-8215-26f420be0ce8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:50.521631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f2eabeb5-695e-47ff-8215-26f420be0ce8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:50.527712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2eabeb5-695e-47ff-8215-26f420be0ce8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:50.528199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2eabeb5-695e-47ff-8215-26f420be0ce8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:50.528622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2eabeb5-695e-47ff-8215-26f420be0ce8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:50.529305 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2eabeb5-695e-47ff-8215-26f420be0ce8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:50.529752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2eabeb5-695e-47ff-8215-26f420be0ce8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:50.530195 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2eabeb5-695e-47ff-8215-26f420be0ce8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:50.666820 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f2eabeb5-695e-47ff-8215-26f420be0ce8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:50.674432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f2eabeb5-695e-47ff-8215-26f420be0ce8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.157477 Sep 08 13:58:50.674995 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 400/677] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:50 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:50.949955 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-da1c7be9-13aa-418c-977e-278b47d7ef4c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:50.950728 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-da1c7be9-13aa-418c-977e-278b47d7ef4c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:50.953763 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=98,nova_api:DELETE=1,nova_api:UPDATE=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:50.957243 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da1c7be9-13aa-418c-977e-278b47d7ef4c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:50.957733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da1c7be9-13aa-418c-977e-278b47d7ef4c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:50.958191 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da1c7be9-13aa-418c-977e-278b47d7ef4c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:50.964634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da1c7be9-13aa-418c-977e-278b47d7ef4c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:50.964634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da1c7be9-13aa-418c-977e-278b47d7ef4c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:50.964634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da1c7be9-13aa-418c-977e-278b47d7ef4c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:51.101892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-da1c7be9-13aa-418c-977e-278b47d7ef4c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:51.109749 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-da1c7be9-13aa-418c-977e-278b47d7ef4c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.163595 Sep 08 13:58:51.110997 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 278/678] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:50 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:51.253124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-74ec0831-9110-4dd1-9f62-4ac23505504b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:51.253727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-74ec0831-9110-4dd1-9f62-4ac23505504b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:51.260386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-74ec0831-9110-4dd1-9f62-4ac23505504b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:51.260871 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-74ec0831-9110-4dd1-9f62-4ac23505504b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:51.261307 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-74ec0831-9110-4dd1-9f62-4ac23505504b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:51.261997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-74ec0831-9110-4dd1-9f62-4ac23505504b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:51.262488 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-74ec0831-9110-4dd1-9f62-4ac23505504b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:51.262911 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-74ec0831-9110-4dd1-9f62-4ac23505504b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:51.393641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-74ec0831-9110-4dd1-9f62-4ac23505504b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:51.401756 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-74ec0831-9110-4dd1-9f62-4ac23505504b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.152689 Sep 08 13:58:51.402341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 401/679] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:51 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:51.697344 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-30da30f0-a599-4348-8ea9-709a95287008 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:51.697950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-30da30f0-a599-4348-8ea9-709a95287008 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:51.705217 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-30da30f0-a599-4348-8ea9-709a95287008 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:51.706065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-30da30f0-a599-4348-8ea9-709a95287008 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:51.706065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-30da30f0-a599-4348-8ea9-709a95287008 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:51.706065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-30da30f0-a599-4348-8ea9-709a95287008 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:51.708183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-30da30f0-a599-4348-8ea9-709a95287008 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:51.708183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-30da30f0-a599-4348-8ea9-709a95287008 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:51.851055 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-30da30f0-a599-4348-8ea9-709a95287008 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:51.854457 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-30da30f0-a599-4348-8ea9-709a95287008 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.160862 Sep 08 13:58:51.855227 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 279/680] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:51 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:52.134047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9e3b7d9d-13b1-4842-8d2d-8dd238d8b642 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:52.134671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9e3b7d9d-13b1-4842-8d2d-8dd238d8b642 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:52.140563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e3b7d9d-13b1-4842-8d2d-8dd238d8b642 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:52.141039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e3b7d9d-13b1-4842-8d2d-8dd238d8b642 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:52.141467 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e3b7d9d-13b1-4842-8d2d-8dd238d8b642 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:52.142165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e3b7d9d-13b1-4842-8d2d-8dd238d8b642 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:52.142611 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e3b7d9d-13b1-4842-8d2d-8dd238d8b642 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:52.143073 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e3b7d9d-13b1-4842-8d2d-8dd238d8b642 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:52.279886 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9e3b7d9d-13b1-4842-8d2d-8dd238d8b642 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:52.287918 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9e3b7d9d-13b1-4842-8d2d-8dd238d8b642 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.157739 Sep 08 13:58:52.288479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 402/681] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:52 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:52.425858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4ce422b1-c6f0-4361-a669-c63f43563985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:52.426472 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4ce422b1-c6f0-4361-a669-c63f43563985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:52.432742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4ce422b1-c6f0-4361-a669-c63f43563985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:52.433218 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4ce422b1-c6f0-4361-a669-c63f43563985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:52.433650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4ce422b1-c6f0-4361-a669-c63f43563985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:52.434365 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4ce422b1-c6f0-4361-a669-c63f43563985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:52.434816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4ce422b1-c6f0-4361-a669-c63f43563985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:52.435233 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4ce422b1-c6f0-4361-a669-c63f43563985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:52.574703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4ce422b1-c6f0-4361-a669-c63f43563985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:52.583211 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4ce422b1-c6f0-4361-a669-c63f43563985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.162082 Sep 08 13:58:52.583780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 280/682] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:52 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:52.878674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c5327f49-b783-4e4a-aae0-df8e14875592 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:52.879318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c5327f49-b783-4e4a-aae0-df8e14875592 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:52.881506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:52.886616 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5327f49-b783-4e4a-aae0-df8e14875592 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:52.886928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5327f49-b783-4e4a-aae0-df8e14875592 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:52.893504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5327f49-b783-4e4a-aae0-df8e14875592 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:52.894515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5327f49-b783-4e4a-aae0-df8e14875592 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:52.895004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5327f49-b783-4e4a-aae0-df8e14875592 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:52.895443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5327f49-b783-4e4a-aae0-df8e14875592 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:53.028748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c5327f49-b783-4e4a-aae0-df8e14875592 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:53.037320 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c5327f49-b783-4e4a-aae0-df8e14875592 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.162606 Sep 08 13:58:53.037914 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 403/683] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:52 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:53.311859 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-96bf38f3-a9b3-44b6-a929-94fab0ff28cd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:53.312372 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-96bf38f3-a9b3-44b6-a929-94fab0ff28cd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:53.319553 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-96bf38f3-a9b3-44b6-a929-94fab0ff28cd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:53.320027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-96bf38f3-a9b3-44b6-a929-94fab0ff28cd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:53.320453 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-96bf38f3-a9b3-44b6-a929-94fab0ff28cd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:53.321138 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-96bf38f3-a9b3-44b6-a929-94fab0ff28cd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:53.321584 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-96bf38f3-a9b3-44b6-a929-94fab0ff28cd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:53.321997 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-96bf38f3-a9b3-44b6-a929-94fab0ff28cd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:53.341507 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:53.457902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-96bf38f3-a9b3-44b6-a929-94fab0ff28cd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:53.464415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-96bf38f3-a9b3-44b6-a929-94fab0ff28cd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.156518 Sep 08 13:58:53.464988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 281/684] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:53 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:53.607303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-63dd2191-338c-4fdf-9d66-689edfa192f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:53.607905 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-63dd2191-338c-4fdf-9d66-689edfa192f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:53.614798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-63dd2191-338c-4fdf-9d66-689edfa192f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:53.615280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-63dd2191-338c-4fdf-9d66-689edfa192f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:53.615718 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-63dd2191-338c-4fdf-9d66-689edfa192f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:53.616405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-63dd2191-338c-4fdf-9d66-689edfa192f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:53.616860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-63dd2191-338c-4fdf-9d66-689edfa192f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:53.617289 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-63dd2191-338c-4fdf-9d66-689edfa192f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:53.750847 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-63dd2191-338c-4fdf-9d66-689edfa192f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:53.759985 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-63dd2191-338c-4fdf-9d66-689edfa192f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.156492 Sep 08 13:58:53.760552 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 404/685] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:53 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:54.059873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0ac5a484-4d83-4e8b-8f4d-760242b54d30 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:54.060499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0ac5a484-4d83-4e8b-8f4d-760242b54d30 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:54.066556 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ac5a484-4d83-4e8b-8f4d-760242b54d30 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:54.067045 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ac5a484-4d83-4e8b-8f4d-760242b54d30 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:54.067477 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ac5a484-4d83-4e8b-8f4d-760242b54d30 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:54.068167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ac5a484-4d83-4e8b-8f4d-760242b54d30 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:54.068617 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ac5a484-4d83-4e8b-8f4d-760242b54d30 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:54.069035 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ac5a484-4d83-4e8b-8f4d-760242b54d30 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:54.201572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0ac5a484-4d83-4e8b-8f4d-760242b54d30 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:54.209285 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0ac5a484-4d83-4e8b-8f4d-760242b54d30 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.153305 Sep 08 13:58:54.209936 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 282/686] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:54 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:54.487904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5fda67d6-7b84-4c2e-9506-5f01be2d6c9d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:54.488431 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5fda67d6-7b84-4c2e-9506-5f01be2d6c9d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:54.494510 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5fda67d6-7b84-4c2e-9506-5f01be2d6c9d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:54.495001 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5fda67d6-7b84-4c2e-9506-5f01be2d6c9d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:54.495439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5fda67d6-7b84-4c2e-9506-5f01be2d6c9d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:54.496129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5fda67d6-7b84-4c2e-9506-5f01be2d6c9d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:54.496588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5fda67d6-7b84-4c2e-9506-5f01be2d6c9d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:54.497023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5fda67d6-7b84-4c2e-9506-5f01be2d6c9d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:54.632432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5fda67d6-7b84-4c2e-9506-5f01be2d6c9d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:54.640220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5fda67d6-7b84-4c2e-9506-5f01be2d6c9d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.156279 Sep 08 13:58:54.640751 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 405/687] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:54 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:54.783387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9f9987c9-41c8-4d0d-b7de-66e087e9392f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:54.783945 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9f9987c9-41c8-4d0d-b7de-66e087e9392f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:54.790618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9f9987c9-41c8-4d0d-b7de-66e087e9392f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:54.791098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9f9987c9-41c8-4d0d-b7de-66e087e9392f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:54.791524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9f9987c9-41c8-4d0d-b7de-66e087e9392f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:54.792213 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9f9987c9-41c8-4d0d-b7de-66e087e9392f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:54.792656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9f9987c9-41c8-4d0d-b7de-66e087e9392f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:54.793075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9f9987c9-41c8-4d0d-b7de-66e087e9392f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:54.932126 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9f9987c9-41c8-4d0d-b7de-66e087e9392f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:54.940954 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9f9987c9-41c8-4d0d-b7de-66e087e9392f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.161708 Sep 08 13:58:54.941733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 283/688] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:54 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:55.233021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6c0a186b-65d0-4f1e-aa5e-dd1b983c3dda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:55.233629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6c0a186b-65d0-4f1e-aa5e-dd1b983c3dda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:55.240624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c0a186b-65d0-4f1e-aa5e-dd1b983c3dda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:55.241075 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c0a186b-65d0-4f1e-aa5e-dd1b983c3dda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:55.241502 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c0a186b-65d0-4f1e-aa5e-dd1b983c3dda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:55.242292 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c0a186b-65d0-4f1e-aa5e-dd1b983c3dda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:55.242869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c0a186b-65d0-4f1e-aa5e-dd1b983c3dda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:55.243556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c0a186b-65d0-4f1e-aa5e-dd1b983c3dda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:55.380252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6c0a186b-65d0-4f1e-aa5e-dd1b983c3dda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:55.382465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:55.388448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6c0a186b-65d0-4f1e-aa5e-dd1b983c3dda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.159598 Sep 08 13:58:55.394567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 406/689] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:55 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:55.665966 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9f5ae20c-8a8f-45cb-936e-0f0f5c207010 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:55.666611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9f5ae20c-8a8f-45cb-936e-0f0f5c207010 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:55.672623 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9f5ae20c-8a8f-45cb-936e-0f0f5c207010 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:55.673106 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9f5ae20c-8a8f-45cb-936e-0f0f5c207010 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:55.673546 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9f5ae20c-8a8f-45cb-936e-0f0f5c207010 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:55.674263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9f5ae20c-8a8f-45cb-936e-0f0f5c207010 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:55.675367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9f5ae20c-8a8f-45cb-936e-0f0f5c207010 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:55.675798 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9f5ae20c-8a8f-45cb-936e-0f0f5c207010 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:55.807307 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9f5ae20c-8a8f-45cb-936e-0f0f5c207010 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:55.815097 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9f5ae20c-8a8f-45cb-936e-0f0f5c207010 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.153646 Sep 08 13:58:55.815663 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 284/690] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:55 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:55.963988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d1082776-d178-4ca8-9816-6555ab24d2d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:55.964599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d1082776-d178-4ca8-9816-6555ab24d2d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:55.971955 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d1082776-d178-4ca8-9816-6555ab24d2d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:55.972573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d1082776-d178-4ca8-9816-6555ab24d2d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:55.973469 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d1082776-d178-4ca8-9816-6555ab24d2d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:55.973613 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d1082776-d178-4ca8-9816-6555ab24d2d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:55.974058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d1082776-d178-4ca8-9816-6555ab24d2d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:55.974500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d1082776-d178-4ca8-9816-6555ab24d2d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:56.115654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d1082776-d178-4ca8-9816-6555ab24d2d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:56.124410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d1082776-d178-4ca8-9816-6555ab24d2d9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.164092 Sep 08 13:58:56.125018 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 407/691] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:55 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:56.414722 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bc8839d8-a2ab-4a42-8ecc-1a9a313b10e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:56.415327 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bc8839d8-a2ab-4a42-8ecc-1a9a313b10e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:56.421262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc8839d8-a2ab-4a42-8ecc-1a9a313b10e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:56.421749 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc8839d8-a2ab-4a42-8ecc-1a9a313b10e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:56.422203 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc8839d8-a2ab-4a42-8ecc-1a9a313b10e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:56.422886 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc8839d8-a2ab-4a42-8ecc-1a9a313b10e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:56.423339 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc8839d8-a2ab-4a42-8ecc-1a9a313b10e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:56.423786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc8839d8-a2ab-4a42-8ecc-1a9a313b10e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:56.559974 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-bc8839d8-a2ab-4a42-8ecc-1a9a313b10e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:56.568333 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bc8839d8-a2ab-4a42-8ecc-1a9a313b10e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.157490 Sep 08 13:58:56.569088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 285/692] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:56 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:56.839219 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-25fd34d9-d426-49d9-99fb-92c89446c157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:56.839823 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-25fd34d9-d426-49d9-99fb-92c89446c157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:56.846954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25fd34d9-d426-49d9-99fb-92c89446c157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:56.847499 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25fd34d9-d426-49d9-99fb-92c89446c157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:56.847938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25fd34d9-d426-49d9-99fb-92c89446c157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:56.848628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25fd34d9-d426-49d9-99fb-92c89446c157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:56.849081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25fd34d9-d426-49d9-99fb-92c89446c157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:56.849504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25fd34d9-d426-49d9-99fb-92c89446c157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:56.983838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-25fd34d9-d426-49d9-99fb-92c89446c157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:56.991868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-25fd34d9-d426-49d9-99fb-92c89446c157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.156536 Sep 08 13:58:56.992447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 408/693] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:56 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:57.148693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3b32cb4a-48f6-4110-8f59-b1a32948c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:57.149296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3b32cb4a-48f6-4110-8f59-b1a32948c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:57.155690 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b32cb4a-48f6-4110-8f59-b1a32948c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:57.156164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b32cb4a-48f6-4110-8f59-b1a32948c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:57.156595 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b32cb4a-48f6-4110-8f59-b1a32948c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:57.157274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b32cb4a-48f6-4110-8f59-b1a32948c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:57.157723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b32cb4a-48f6-4110-8f59-b1a32948c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:57.158482 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b32cb4a-48f6-4110-8f59-b1a32948c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:57.289221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3b32cb4a-48f6-4110-8f59-b1a32948c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:57.297785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3b32cb4a-48f6-4110-8f59-b1a32948c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.152939 Sep 08 13:58:57.298381 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 286/694] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:57 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:57.591459 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4fedc533-04c6-4e69-a300-de6d2bfce4f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:57.592058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4fedc533-04c6-4e69-a300-de6d2bfce4f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:57.598194 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fedc533-04c6-4e69-a300-de6d2bfce4f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:57.598684 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fedc533-04c6-4e69-a300-de6d2bfce4f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:57.599118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fedc533-04c6-4e69-a300-de6d2bfce4f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:57.601125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fedc533-04c6-4e69-a300-de6d2bfce4f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:57.601125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fedc533-04c6-4e69-a300-de6d2bfce4f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:57.601125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fedc533-04c6-4e69-a300-de6d2bfce4f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:57.737331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4fedc533-04c6-4e69-a300-de6d2bfce4f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:57.753235 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4fedc533-04c6-4e69-a300-de6d2bfce4f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.165699 Sep 08 13:58:57.753785 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 409/695] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:57 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:58.016369 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a7d25a03-ed2a-420e-81af-a3014cdbd2a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:58.016983 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a7d25a03-ed2a-420e-81af-a3014cdbd2a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:58.023602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7d25a03-ed2a-420e-81af-a3014cdbd2a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:58.024093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7d25a03-ed2a-420e-81af-a3014cdbd2a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:58.024532 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7d25a03-ed2a-420e-81af-a3014cdbd2a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:58.025237 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7d25a03-ed2a-420e-81af-a3014cdbd2a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:58.025690 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7d25a03-ed2a-420e-81af-a3014cdbd2a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:58.026118 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7d25a03-ed2a-420e-81af-a3014cdbd2a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:58.160606 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a7d25a03-ed2a-420e-81af-a3014cdbd2a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:58.168347 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a7d25a03-ed2a-420e-81af-a3014cdbd2a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.156742 Sep 08 13:58:58.168916 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 287/696] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:58 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:58.323435 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7cb1a547-f0d7-4683-8862-3e1c07467c05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:58.324172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7cb1a547-f0d7-4683-8862-3e1c07467c05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:58.337794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7cb1a547-f0d7-4683-8862-3e1c07467c05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:58.338554 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7cb1a547-f0d7-4683-8862-3e1c07467c05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:58.338899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7cb1a547-f0d7-4683-8862-3e1c07467c05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:58.339614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7cb1a547-f0d7-4683-8862-3e1c07467c05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:58.340117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7cb1a547-f0d7-4683-8862-3e1c07467c05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:58.340545 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7cb1a547-f0d7-4683-8862-3e1c07467c05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:58.472474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7cb1a547-f0d7-4683-8862-3e1c07467c05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:58.481367 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7cb1a547-f0d7-4683-8862-3e1c07467c05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.163677 Sep 08 13:58:58.481940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 410/697] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:58 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:58.775300 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f80cfb7b-11d1-41d2-8e1f-fb23ffda4af2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:58.775939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f80cfb7b-11d1-41d2-8e1f-fb23ffda4af2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:58.781920 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f80cfb7b-11d1-41d2-8e1f-fb23ffda4af2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:58.782441 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f80cfb7b-11d1-41d2-8e1f-fb23ffda4af2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:58.782879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f80cfb7b-11d1-41d2-8e1f-fb23ffda4af2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:58.783830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f80cfb7b-11d1-41d2-8e1f-fb23ffda4af2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:58.784336 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f80cfb7b-11d1-41d2-8e1f-fb23ffda4af2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:58.784770 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f80cfb7b-11d1-41d2-8e1f-fb23ffda4af2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:58.928102 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f80cfb7b-11d1-41d2-8e1f-fb23ffda4af2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:58.935886 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f80cfb7b-11d1-41d2-8e1f-fb23ffda4af2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.164459 Sep 08 13:58:58.936477 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 288/698] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:58 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:58:59.192863 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9bf48bf1-cc42-4595-9efa-09c9772b7b6e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:59.193393 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9bf48bf1-cc42-4595-9efa-09c9772b7b6e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:59.199385 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bf48bf1-cc42-4595-9efa-09c9772b7b6e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:59.199867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bf48bf1-cc42-4595-9efa-09c9772b7b6e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:59.200305 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bf48bf1-cc42-4595-9efa-09c9772b7b6e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:59.200994 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bf48bf1-cc42-4595-9efa-09c9772b7b6e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:59.201451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bf48bf1-cc42-4595-9efa-09c9772b7b6e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:59.201878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bf48bf1-cc42-4595-9efa-09c9772b7b6e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:59.330652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9bf48bf1-cc42-4595-9efa-09c9772b7b6e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:59.339590 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9bf48bf1-cc42-4595-9efa-09c9772b7b6e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.151483 Sep 08 13:58:59.340159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 411/699] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:59 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:59.505786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f4a41616-a350-4803-a3e0-0312ee9cb1cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:59.506371 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f4a41616-a350-4803-a3e0-0312ee9cb1cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:59.512884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f4a41616-a350-4803-a3e0-0312ee9cb1cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:59.520167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f4a41616-a350-4803-a3e0-0312ee9cb1cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:59.520167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f4a41616-a350-4803-a3e0-0312ee9cb1cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:59.520167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f4a41616-a350-4803-a3e0-0312ee9cb1cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:59.520167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f4a41616-a350-4803-a3e0-0312ee9cb1cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:59.520534 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f4a41616-a350-4803-a3e0-0312ee9cb1cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:59.649241 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f4a41616-a350-4803-a3e0-0312ee9cb1cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:58:59.657479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f4a41616-a350-4803-a3e0-0312ee9cb1cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.156284 Sep 08 13:58:59.658446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 289/700] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:59 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:58:59.958006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6adc1c6f-55af-45d7-974e-9c10c37fba61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:58:59.958654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6adc1c6f-55af-45d7-974e-9c10c37fba61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:58:59.964855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6adc1c6f-55af-45d7-974e-9c10c37fba61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:59.965339 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6adc1c6f-55af-45d7-974e-9c10c37fba61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:59.965775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6adc1c6f-55af-45d7-974e-9c10c37fba61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:58:59.966543 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6adc1c6f-55af-45d7-974e-9c10c37fba61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:58:59.966992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6adc1c6f-55af-45d7-974e-9c10c37fba61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:58:59.967412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6adc1c6f-55af-45d7-974e-9c10c37fba61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:00.104924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6adc1c6f-55af-45d7-974e-9c10c37fba61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:00.112684 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6adc1c6f-55af-45d7-974e-9c10c37fba61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.158473 Sep 08 13:59:00.113917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 412/701] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:58:59 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:00.363119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-168acb02-0858-4c8f-a18d-8c59c0a23157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:00.364370 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-168acb02-0858-4c8f-a18d-8c59c0a23157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:00.369886 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-168acb02-0858-4c8f-a18d-8c59c0a23157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:00.370404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-168acb02-0858-4c8f-a18d-8c59c0a23157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:00.370836 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-168acb02-0858-4c8f-a18d-8c59c0a23157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:00.379728 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-168acb02-0858-4c8f-a18d-8c59c0a23157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:00.380301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-168acb02-0858-4c8f-a18d-8c59c0a23157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:00.380736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-168acb02-0858-4c8f-a18d-8c59c0a23157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:00.516815 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-168acb02-0858-4c8f-a18d-8c59c0a23157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:00.525205 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-168acb02-0858-4c8f-a18d-8c59c0a23157 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.166080 Sep 08 13:59:00.525926 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 290/702] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:00 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:00.681167 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3510507d-1068-430e-9356-b444fc294985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:00.681773 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3510507d-1068-430e-9356-b444fc294985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:00.688319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3510507d-1068-430e-9356-b444fc294985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:00.690874 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3510507d-1068-430e-9356-b444fc294985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:00.690874 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3510507d-1068-430e-9356-b444fc294985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:00.690874 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3510507d-1068-430e-9356-b444fc294985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:00.690874 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3510507d-1068-430e-9356-b444fc294985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:00.691309 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3510507d-1068-430e-9356-b444fc294985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:00.821671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3510507d-1068-430e-9356-b444fc294985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:00.829798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3510507d-1068-430e-9356-b444fc294985 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.152637 Sep 08 13:59:00.830389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 413/703] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:00 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:59:01.136123 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e1019b6a-23ba-4c25-a045-d68b6f4a5cdb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:01.136767 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e1019b6a-23ba-4c25-a045-d68b6f4a5cdb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:01.143251 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1019b6a-23ba-4c25-a045-d68b6f4a5cdb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:01.143734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1019b6a-23ba-4c25-a045-d68b6f4a5cdb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:01.144170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1019b6a-23ba-4c25-a045-d68b6f4a5cdb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:01.144896 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1019b6a-23ba-4c25-a045-d68b6f4a5cdb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:01.145347 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1019b6a-23ba-4c25-a045-d68b6f4a5cdb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:01.145775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1019b6a-23ba-4c25-a045-d68b6f4a5cdb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:01.276053 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e1019b6a-23ba-4c25-a045-d68b6f4a5cdb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:01.283981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e1019b6a-23ba-4c25-a045-d68b6f4a5cdb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.151647 Sep 08 13:59:01.284545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 291/704] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:01 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:01.548842 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d985d419-5ebd-4fe8-b157-c152793b31fa tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:01.549442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d985d419-5ebd-4fe8-b157-c152793b31fa tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:01.556090 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d985d419-5ebd-4fe8-b157-c152793b31fa tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:01.556565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d985d419-5ebd-4fe8-b157-c152793b31fa tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:01.556996 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d985d419-5ebd-4fe8-b157-c152793b31fa tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:01.557676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d985d419-5ebd-4fe8-b157-c152793b31fa tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:01.558123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d985d419-5ebd-4fe8-b157-c152793b31fa tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:01.558566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d985d419-5ebd-4fe8-b157-c152793b31fa tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:01.693137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d985d419-5ebd-4fe8-b157-c152793b31fa tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:01.700986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d985d419-5ebd-4fe8-b157-c152793b31fa tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.156049 Sep 08 13:59:01.701546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 414/705] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:01 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:01.853397 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-94c28ac7-70a9-4f9f-8f91-1f6c848891bf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:01.854026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-94c28ac7-70a9-4f9f-8f91-1f6c848891bf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:01.860624 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94c28ac7-70a9-4f9f-8f91-1f6c848891bf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:01.861106 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94c28ac7-70a9-4f9f-8f91-1f6c848891bf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:01.861530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94c28ac7-70a9-4f9f-8f91-1f6c848891bf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:01.862235 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94c28ac7-70a9-4f9f-8f91-1f6c848891bf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:01.862686 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94c28ac7-70a9-4f9f-8f91-1f6c848891bf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:01.863102 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94c28ac7-70a9-4f9f-8f91-1f6c848891bf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:01.997294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-94c28ac7-70a9-4f9f-8f91-1f6c848891bf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:02.005708 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-94c28ac7-70a9-4f9f-8f91-1f6c848891bf tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.156154 Sep 08 13:59:02.006308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 292/706] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:01 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:02.306329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cdc7bf55-686c-4932-8796-854f3d6bdbda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:02.306929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cdc7bf55-686c-4932-8796-854f3d6bdbda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:02.312813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cdc7bf55-686c-4932-8796-854f3d6bdbda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:02.313436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cdc7bf55-686c-4932-8796-854f3d6bdbda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:02.313718 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cdc7bf55-686c-4932-8796-854f3d6bdbda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:02.314432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cdc7bf55-686c-4932-8796-854f3d6bdbda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:02.314884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cdc7bf55-686c-4932-8796-854f3d6bdbda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:02.315327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cdc7bf55-686c-4932-8796-854f3d6bdbda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:02.449165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cdc7bf55-686c-4932-8796-854f3d6bdbda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:02.456802 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cdc7bf55-686c-4932-8796-854f3d6bdbda tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.154365 Sep 08 13:59:02.457355 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 415/707] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:02 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:59:02.724883 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e3becd34-963c-4f24-9300-a844c9f48102 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:02.725414 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e3becd34-963c-4f24-9300-a844c9f48102 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:02.731451 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e3becd34-963c-4f24-9300-a844c9f48102 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:02.731938 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e3becd34-963c-4f24-9300-a844c9f48102 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:02.732377 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e3becd34-963c-4f24-9300-a844c9f48102 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:02.733061 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e3becd34-963c-4f24-9300-a844c9f48102 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:02.733522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e3becd34-963c-4f24-9300-a844c9f48102 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:02.733948 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e3becd34-963c-4f24-9300-a844c9f48102 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:02.861029 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e3becd34-963c-4f24-9300-a844c9f48102 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:02.868736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e3becd34-963c-4f24-9300-a844c9f48102 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.148336 Sep 08 13:59:02.869304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 293/708] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:02 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:03.029135 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f14336fc-ff5d-4203-801f-a0e9750a54cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:03.029675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f14336fc-ff5d-4203-801f-a0e9750a54cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:03.036862 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f14336fc-ff5d-4203-801f-a0e9750a54cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:03.037348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f14336fc-ff5d-4203-801f-a0e9750a54cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:03.037785 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f14336fc-ff5d-4203-801f-a0e9750a54cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:03.038514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f14336fc-ff5d-4203-801f-a0e9750a54cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:03.039024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f14336fc-ff5d-4203-801f-a0e9750a54cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:03.039447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f14336fc-ff5d-4203-801f-a0e9750a54cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:03.174006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f14336fc-ff5d-4203-801f-a0e9750a54cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:03.182643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f14336fc-ff5d-4203-801f-a0e9750a54cc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.157341 Sep 08 13:59:03.183376 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 416/709] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:03 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:03.479447 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-00fbd955-315d-4999-bd4f-915b6f1a81ac tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:03.480047 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-00fbd955-315d-4999-bd4f-915b6f1a81ac tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:03.485971 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00fbd955-315d-4999-bd4f-915b6f1a81ac tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:03.486413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00fbd955-315d-4999-bd4f-915b6f1a81ac tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:03.486843 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00fbd955-315d-4999-bd4f-915b6f1a81ac tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:03.487520 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00fbd955-315d-4999-bd4f-915b6f1a81ac tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:03.487962 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00fbd955-315d-4999-bd4f-915b6f1a81ac tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:03.488386 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00fbd955-315d-4999-bd4f-915b6f1a81ac tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:03.625757 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-00fbd955-315d-4999-bd4f-915b6f1a81ac tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:03.633659 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-00fbd955-315d-4999-bd4f-915b6f1a81ac tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.158196 Sep 08 13:59:03.646075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 294/710] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:03 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:03.892887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6dfa11b3-0791-4e80-b7b8-3785bfbb088e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:03.893531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6dfa11b3-0791-4e80-b7b8-3785bfbb088e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:03.900239 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6dfa11b3-0791-4e80-b7b8-3785bfbb088e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:03.900731 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6dfa11b3-0791-4e80-b7b8-3785bfbb088e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:03.901193 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6dfa11b3-0791-4e80-b7b8-3785bfbb088e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:03.901869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6dfa11b3-0791-4e80-b7b8-3785bfbb088e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:03.902357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6dfa11b3-0791-4e80-b7b8-3785bfbb088e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:03.902785 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6dfa11b3-0791-4e80-b7b8-3785bfbb088e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:04.031015 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6dfa11b3-0791-4e80-b7b8-3785bfbb088e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:04.039546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6dfa11b3-0791-4e80-b7b8-3785bfbb088e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.150750 Sep 08 13:59:04.040166 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 417/711] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:03 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:04.206118 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-76bdf293-c968-468d-90a9-74790e6452d2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:04.206756 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-76bdf293-c968-468d-90a9-74790e6452d2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:04.213528 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-76bdf293-c968-468d-90a9-74790e6452d2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:04.214016 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-76bdf293-c968-468d-90a9-74790e6452d2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:04.214491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-76bdf293-c968-468d-90a9-74790e6452d2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:04.215187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-76bdf293-c968-468d-90a9-74790e6452d2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:04.215667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-76bdf293-c968-468d-90a9-74790e6452d2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:04.216089 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-76bdf293-c968-468d-90a9-74790e6452d2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:04.348060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-76bdf293-c968-468d-90a9-74790e6452d2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:04.356932 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-76bdf293-c968-468d-90a9-74790e6452d2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.154629 Sep 08 13:59:04.357502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 295/712] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:04 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:04.656846 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5abff4e8-c92d-43e1-9986-85307c471ff7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:04.657493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5abff4e8-c92d-43e1-9986-85307c471ff7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:04.663834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5abff4e8-c92d-43e1-9986-85307c471ff7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:04.664324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5abff4e8-c92d-43e1-9986-85307c471ff7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:04.664826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5abff4e8-c92d-43e1-9986-85307c471ff7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:04.665526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5abff4e8-c92d-43e1-9986-85307c471ff7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:04.665986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5abff4e8-c92d-43e1-9986-85307c471ff7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:04.666454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5abff4e8-c92d-43e1-9986-85307c471ff7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:04.797956 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5abff4e8-c92d-43e1-9986-85307c471ff7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:04.806479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5abff4e8-c92d-43e1-9986-85307c471ff7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.153485 Sep 08 13:59:04.807045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 418/713] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:04 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:05.063564 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a14d157f-fe10-4c80-a2ed-4bc25963524e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:05.064092 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a14d157f-fe10-4c80-a2ed-4bc25963524e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:05.070880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a14d157f-fe10-4c80-a2ed-4bc25963524e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:05.071361 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a14d157f-fe10-4c80-a2ed-4bc25963524e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:05.071802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a14d157f-fe10-4c80-a2ed-4bc25963524e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:05.072483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a14d157f-fe10-4c80-a2ed-4bc25963524e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:05.072931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a14d157f-fe10-4c80-a2ed-4bc25963524e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:05.073352 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a14d157f-fe10-4c80-a2ed-4bc25963524e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:05.203347 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a14d157f-fe10-4c80-a2ed-4bc25963524e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:05.211682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a14d157f-fe10-4c80-a2ed-4bc25963524e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.152129 Sep 08 13:59:05.212375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 296/714] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:05 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:05.380880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8c6c2168-f84a-4ff9-83b4-39336596e080 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:05.381481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8c6c2168-f84a-4ff9-83b4-39336596e080 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:05.387975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c6c2168-f84a-4ff9-83b4-39336596e080 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:05.388458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c6c2168-f84a-4ff9-83b4-39336596e080 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:05.388912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c6c2168-f84a-4ff9-83b4-39336596e080 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:05.389578 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c6c2168-f84a-4ff9-83b4-39336596e080 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:05.390027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c6c2168-f84a-4ff9-83b4-39336596e080 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:05.390482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c6c2168-f84a-4ff9-83b4-39336596e080 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:05.524130 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8c6c2168-f84a-4ff9-83b4-39336596e080 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:05.532633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8c6c2168-f84a-4ff9-83b4-39336596e080 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.155646 Sep 08 13:59:05.533206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 419/715] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:05 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:05.828729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-070ad970-3868-493e-a802-b288695a4901 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:05.829328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-070ad970-3868-493e-a802-b288695a4901 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:05.835978 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-070ad970-3868-493e-a802-b288695a4901 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:05.836448 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-070ad970-3868-493e-a802-b288695a4901 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:05.836871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-070ad970-3868-493e-a802-b288695a4901 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:05.837545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-070ad970-3868-493e-a802-b288695a4901 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:05.838334 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-070ad970-3868-493e-a802-b288695a4901 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:05.838476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-070ad970-3868-493e-a802-b288695a4901 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:05.973444 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-070ad970-3868-493e-a802-b288695a4901 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:05.981241 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-070ad970-3868-493e-a802-b288695a4901 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.156335 Sep 08 13:59:05.981807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 297/716] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:05 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:06.236083 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-16f454cd-e854-462e-837a-882a895d0824 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:06.236600 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-16f454cd-e854-462e-837a-882a895d0824 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:06.242612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-16f454cd-e854-462e-837a-882a895d0824 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:06.243099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-16f454cd-e854-462e-837a-882a895d0824 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:06.243568 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-16f454cd-e854-462e-837a-882a895d0824 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:06.244256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-16f454cd-e854-462e-837a-882a895d0824 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:06.244733 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-16f454cd-e854-462e-837a-882a895d0824 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:06.245643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-16f454cd-e854-462e-837a-882a895d0824 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:06.374180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-16f454cd-e854-462e-837a-882a895d0824 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:06.382035 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-16f454cd-e854-462e-837a-882a895d0824 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.150063 Sep 08 13:59:06.382646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 420/717] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:06 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:06.558487 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ecca3fa6-5823-452f-b233-0ec0c5704e7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:06.559565 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ecca3fa6-5823-452f-b233-0ec0c5704e7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:06.566105 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ecca3fa6-5823-452f-b233-0ec0c5704e7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:06.566608 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ecca3fa6-5823-452f-b233-0ec0c5704e7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:06.567036 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ecca3fa6-5823-452f-b233-0ec0c5704e7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:06.567713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ecca3fa6-5823-452f-b233-0ec0c5704e7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:06.568156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ecca3fa6-5823-452f-b233-0ec0c5704e7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:06.568574 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ecca3fa6-5823-452f-b233-0ec0c5704e7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:06.572469 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:06.700609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ecca3fa6-5823-452f-b233-0ec0c5704e7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:06.708749 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ecca3fa6-5823-452f-b233-0ec0c5704e7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.154434 Sep 08 13:59:06.709310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 298/718] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:06 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:07.004245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b1ec65eb-5243-4bfc-a069-d2e9db3e4353 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:07.004877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b1ec65eb-5243-4bfc-a069-d2e9db3e4353 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:07.011423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b1ec65eb-5243-4bfc-a069-d2e9db3e4353 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:07.011914 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b1ec65eb-5243-4bfc-a069-d2e9db3e4353 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:07.012386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b1ec65eb-5243-4bfc-a069-d2e9db3e4353 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:07.013083 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b1ec65eb-5243-4bfc-a069-d2e9db3e4353 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:07.013542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b1ec65eb-5243-4bfc-a069-d2e9db3e4353 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:07.013968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b1ec65eb-5243-4bfc-a069-d2e9db3e4353 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:07.147234 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b1ec65eb-5243-4bfc-a069-d2e9db3e4353 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:07.155375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b1ec65eb-5243-4bfc-a069-d2e9db3e4353 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.155068 Sep 08 13:59:07.155937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 421/719] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:06 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:07.405776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e48bd0f5-a001-453b-b595-994d6f7db8e9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:07.406380 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e48bd0f5-a001-453b-b595-994d6f7db8e9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:07.412907 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e48bd0f5-a001-453b-b595-994d6f7db8e9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:07.413401 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e48bd0f5-a001-453b-b595-994d6f7db8e9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:07.413836 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e48bd0f5-a001-453b-b595-994d6f7db8e9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:07.414567 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e48bd0f5-a001-453b-b595-994d6f7db8e9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:07.415043 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e48bd0f5-a001-453b-b595-994d6f7db8e9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:07.415468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e48bd0f5-a001-453b-b595-994d6f7db8e9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:07.546801 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e48bd0f5-a001-453b-b595-994d6f7db8e9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:07.556989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e48bd0f5-a001-453b-b595-994d6f7db8e9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.155687 Sep 08 13:59:07.557585 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 299/720] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:07 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:07.732158 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-df02ff12-c110-4854-8c2e-36256fe5c31c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:07.732717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-df02ff12-c110-4854-8c2e-36256fe5c31c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:07.738714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df02ff12-c110-4854-8c2e-36256fe5c31c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:07.745757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df02ff12-c110-4854-8c2e-36256fe5c31c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:07.745757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df02ff12-c110-4854-8c2e-36256fe5c31c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:07.745757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df02ff12-c110-4854-8c2e-36256fe5c31c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:07.745757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df02ff12-c110-4854-8c2e-36256fe5c31c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:07.746132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df02ff12-c110-4854-8c2e-36256fe5c31c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:07.870022 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-df02ff12-c110-4854-8c2e-36256fe5c31c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:07.879068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-df02ff12-c110-4854-8c2e-36256fe5c31c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.151144 Sep 08 13:59:07.879641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 422/721] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:07 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:08.178868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d98fd071-5040-42b0-a28d-af2a81fa024b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:08.179504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d98fd071-5040-42b0-a28d-af2a81fa024b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:08.186934 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d98fd071-5040-42b0-a28d-af2a81fa024b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:08.187419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d98fd071-5040-42b0-a28d-af2a81fa024b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:08.187854 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d98fd071-5040-42b0-a28d-af2a81fa024b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:08.188547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d98fd071-5040-42b0-a28d-af2a81fa024b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:08.189002 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d98fd071-5040-42b0-a28d-af2a81fa024b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:08.189424 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d98fd071-5040-42b0-a28d-af2a81fa024b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:08.322591 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d98fd071-5040-42b0-a28d-af2a81fa024b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:08.330777 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d98fd071-5040-42b0-a28d-af2a81fa024b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.155733 Sep 08 13:59:08.331379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 300/722] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:08 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:08.581506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-335553d9-18b0-412e-a0a3-531286f7639c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:08.582016 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-335553d9-18b0-412e-a0a3-531286f7639c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:08.588481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-335553d9-18b0-412e-a0a3-531286f7639c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:08.588958 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-335553d9-18b0-412e-a0a3-531286f7639c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:08.589392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-335553d9-18b0-412e-a0a3-531286f7639c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:08.590125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-335553d9-18b0-412e-a0a3-531286f7639c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:08.590599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-335553d9-18b0-412e-a0a3-531286f7639c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:08.591524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-335553d9-18b0-412e-a0a3-531286f7639c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:08.606466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:08.726225 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-335553d9-18b0-412e-a0a3-531286f7639c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:08.733853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-335553d9-18b0-412e-a0a3-531286f7639c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.156359 Sep 08 13:59:08.734475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 423/723] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:08 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:08.902759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b9d30ba8-ed13-41ff-9a21-ba741269d5a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:08.903450 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b9d30ba8-ed13-41ff-9a21-ba741269d5a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:08.911112 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b9d30ba8-ed13-41ff-9a21-ba741269d5a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:08.911596 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b9d30ba8-ed13-41ff-9a21-ba741269d5a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:08.912024 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b9d30ba8-ed13-41ff-9a21-ba741269d5a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:08.912712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b9d30ba8-ed13-41ff-9a21-ba741269d5a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:08.913157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b9d30ba8-ed13-41ff-9a21-ba741269d5a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:08.913573 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b9d30ba8-ed13-41ff-9a21-ba741269d5a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:09.049614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b9d30ba8-ed13-41ff-9a21-ba741269d5a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:09.057910 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b9d30ba8-ed13-41ff-9a21-ba741269d5a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.158983 Sep 08 13:59:09.058509 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 301/724] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:08 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:59:09.355336 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-aa9720f4-df3e-4f11-b62d-18fc3f9a4993 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:09.355930 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-aa9720f4-df3e-4f11-b62d-18fc3f9a4993 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:09.361947 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa9720f4-df3e-4f11-b62d-18fc3f9a4993 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:09.362422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa9720f4-df3e-4f11-b62d-18fc3f9a4993 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:09.362854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa9720f4-df3e-4f11-b62d-18fc3f9a4993 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:09.363540 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa9720f4-df3e-4f11-b62d-18fc3f9a4993 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:09.363990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa9720f4-df3e-4f11-b62d-18fc3f9a4993 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:09.364406 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa9720f4-df3e-4f11-b62d-18fc3f9a4993 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:09.496801 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-aa9720f4-df3e-4f11-b62d-18fc3f9a4993 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:09.505062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-aa9720f4-df3e-4f11-b62d-18fc3f9a4993 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.153568 Sep 08 13:59:09.505617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 424/725] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:09 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:09.757269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-448a48cb-3410-4e08-b8b0-fc63670e08db tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:09.757863 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-448a48cb-3410-4e08-b8b0-fc63670e08db tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:09.764283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-448a48cb-3410-4e08-b8b0-fc63670e08db tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:09.764760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-448a48cb-3410-4e08-b8b0-fc63670e08db tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:09.765188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-448a48cb-3410-4e08-b8b0-fc63670e08db tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:09.765863 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-448a48cb-3410-4e08-b8b0-fc63670e08db tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:09.766343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-448a48cb-3410-4e08-b8b0-fc63670e08db tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:09.766759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-448a48cb-3410-4e08-b8b0-fc63670e08db tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:09.895185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-448a48cb-3410-4e08-b8b0-fc63670e08db tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:09.902786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-448a48cb-3410-4e08-b8b0-fc63670e08db tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.149442 Sep 08 13:59:09.903348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 302/726] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:09 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:10.080833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-57477868-ea94-4e71-80a0-18e8e98e0d6a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:10.081446 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-57477868-ea94-4e71-80a0-18e8e98e0d6a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:10.087989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57477868-ea94-4e71-80a0-18e8e98e0d6a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:10.088470 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57477868-ea94-4e71-80a0-18e8e98e0d6a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:10.088900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57477868-ea94-4e71-80a0-18e8e98e0d6a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:10.089587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57477868-ea94-4e71-80a0-18e8e98e0d6a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:10.090037 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57477868-ea94-4e71-80a0-18e8e98e0d6a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:10.090482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57477868-ea94-4e71-80a0-18e8e98e0d6a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:10.223724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-57477868-ea94-4e71-80a0-18e8e98e0d6a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:10.231967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-57477868-ea94-4e71-80a0-18e8e98e0d6a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.154985 Sep 08 13:59:10.232536 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 425/727] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:10 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:10.535507 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ad1b1a63-bbdf-437f-b121-2ca1336739b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:10.536037 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ad1b1a63-bbdf-437f-b121-2ca1336739b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:10.542241 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ad1b1a63-bbdf-437f-b121-2ca1336739b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:10.542734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ad1b1a63-bbdf-437f-b121-2ca1336739b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:10.543169 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ad1b1a63-bbdf-437f-b121-2ca1336739b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:10.543864 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ad1b1a63-bbdf-437f-b121-2ca1336739b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:10.545035 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ad1b1a63-bbdf-437f-b121-2ca1336739b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:10.545460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ad1b1a63-bbdf-437f-b121-2ca1336739b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:10.679439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ad1b1a63-bbdf-437f-b121-2ca1336739b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:10.687212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ad1b1a63-bbdf-437f-b121-2ca1336739b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.155889 Sep 08 13:59:10.687781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 303/728] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:10 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:10.926694 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-dfa18961-42ca-4e0d-9331-bae6e030aff5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:10.927299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-dfa18961-42ca-4e0d-9331-bae6e030aff5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:10.933248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dfa18961-42ca-4e0d-9331-bae6e030aff5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:10.933729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dfa18961-42ca-4e0d-9331-bae6e030aff5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:10.934181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dfa18961-42ca-4e0d-9331-bae6e030aff5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:10.934877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dfa18961-42ca-4e0d-9331-bae6e030aff5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:10.935328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dfa18961-42ca-4e0d-9331-bae6e030aff5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:10.935755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dfa18961-42ca-4e0d-9331-bae6e030aff5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:11.073031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-dfa18961-42ca-4e0d-9331-bae6e030aff5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:11.082201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-dfa18961-42ca-4e0d-9331-bae6e030aff5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.159302 Sep 08 13:59:11.082941 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 426/729] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:10 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:11.254950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-27f5cd8e-2cfb-45f8-aa37-1041271b41d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:11.255513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-27f5cd8e-2cfb-45f8-aa37-1041271b41d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:11.262066 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27f5cd8e-2cfb-45f8-aa37-1041271b41d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:11.262581 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27f5cd8e-2cfb-45f8-aa37-1041271b41d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:11.263023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27f5cd8e-2cfb-45f8-aa37-1041271b41d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:11.264229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27f5cd8e-2cfb-45f8-aa37-1041271b41d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:11.264681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27f5cd8e-2cfb-45f8-aa37-1041271b41d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:11.265104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27f5cd8e-2cfb-45f8-aa37-1041271b41d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:11.399492 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-27f5cd8e-2cfb-45f8-aa37-1041271b41d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:11.408930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-27f5cd8e-2cfb-45f8-aa37-1041271b41d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.157910 Sep 08 13:59:11.410337 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 304/730] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:11 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:11.709984 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-27eef214-d3e9-433b-a521-7701ca9fedb6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:11.710621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-27eef214-d3e9-433b-a521-7701ca9fedb6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:11.717098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27eef214-d3e9-433b-a521-7701ca9fedb6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:11.717587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27eef214-d3e9-433b-a521-7701ca9fedb6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:11.718034 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27eef214-d3e9-433b-a521-7701ca9fedb6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:11.718751 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27eef214-d3e9-433b-a521-7701ca9fedb6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:11.719209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27eef214-d3e9-433b-a521-7701ca9fedb6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:11.719644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27eef214-d3e9-433b-a521-7701ca9fedb6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:11.851143 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-27eef214-d3e9-433b-a521-7701ca9fedb6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:11.859335 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-27eef214-d3e9-433b-a521-7701ca9fedb6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.153198 Sep 08 13:59:11.859890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 427/731] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:11 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:12.109965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c2fe72e7-47a1-4375-84e3-8ed0c47b1fd3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:12.111262 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c2fe72e7-47a1-4375-84e3-8ed0c47b1fd3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:12.115131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d705475b-2d51-479d-b108-81b4da1d4f23 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:12.115632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d705475b-2d51-479d-b108-81b4da1d4f23 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:12.117352 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2fe72e7-47a1-4375-84e3-8ed0c47b1fd3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:12.117789 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2fe72e7-47a1-4375-84e3-8ed0c47b1fd3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:12.130599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2fe72e7-47a1-4375-84e3-8ed0c47b1fd3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:12.130599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2fe72e7-47a1-4375-84e3-8ed0c47b1fd3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:12.130599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2fe72e7-47a1-4375-84e3-8ed0c47b1fd3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:12.130599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2fe72e7-47a1-4375-84e3-8ed0c47b1fd3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:12.130995 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d705475b-2d51-479d-b108-81b4da1d4f23 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:12.130995 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d705475b-2d51-479d-b108-81b4da1d4f23 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:12.130995 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d705475b-2d51-479d-b108-81b4da1d4f23 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:12.130995 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d705475b-2d51-479d-b108-81b4da1d4f23 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:12.131366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d705475b-2d51-479d-b108-81b4da1d4f23 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:12.131366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d705475b-2d51-479d-b108-81b4da1d4f23 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:12.156604 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d705475b-2d51-479d-b108-81b4da1d4f23 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments/781bf9e8-460d-434e-8629-f5fd94f42731" status: 200 len: 194 microversion: 2.20 time: 0.146978 Sep 08 13:59:12.157193 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 305/732] 10.222.0.25 () {60 vars in 1454 bytes} [Thu Sep 8 13:59:12 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments/781bf9e8-460d-434e-8629-f5fd94f42731 => generated 194 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:12.226011 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-eec368a1-bf47-4a00-b1ab-e69a0cb10196 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:12.226524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-eec368a1-bf47-4a00-b1ab-e69a0cb10196 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:12.232307 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eec368a1-bf47-4a00-b1ab-e69a0cb10196 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:12.232776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eec368a1-bf47-4a00-b1ab-e69a0cb10196 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:12.233215 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eec368a1-bf47-4a00-b1ab-e69a0cb10196 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:12.233879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eec368a1-bf47-4a00-b1ab-e69a0cb10196 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:12.234354 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eec368a1-bf47-4a00-b1ab-e69a0cb10196 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:12.234772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eec368a1-bf47-4a00-b1ab-e69a0cb10196 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:12.259752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c2fe72e7-47a1-4375-84e3-8ed0c47b1fd3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:12.268461 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c2fe72e7-47a1-4375-84e3-8ed0c47b1fd3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.163607 Sep 08 13:59:12.269031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 428/733] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:12 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:59:12.316846 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-eec368a1-bf47-4a00-b1ab-e69a0cb10196 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lazy-loading 'services' on Instance uuid c14f6817-e3ab-4a36-ba23-590498bf1b25 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 13:59:12.432849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-49e619f4-92b0-40b0-aa03-4b58d1ab2e93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:12.433465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-49e619f4-92b0-40b0-aa03-4b58d1ab2e93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:12.440276 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-49e619f4-92b0-40b0-aa03-4b58d1ab2e93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:12.441117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-49e619f4-92b0-40b0-aa03-4b58d1ab2e93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:12.441230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-49e619f4-92b0-40b0-aa03-4b58d1ab2e93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:12.441860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-49e619f4-92b0-40b0-aa03-4b58d1ab2e93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:12.442359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-49e619f4-92b0-40b0-aa03-4b58d1ab2e93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:12.442789 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-49e619f4-92b0-40b0-aa03-4b58d1ab2e93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:12.538631 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-eec368a1-bf47-4a00-b1ab-e69a0cb10196 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments/781bf9e8-460d-434e-8629-f5fd94f42731" status: 202 len: 0 microversion: 2.20 time: 0.316411 Sep 08 13:59:12.539262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 306/734] 10.222.0.25 () {60 vars in 1457 bytes} [Thu Sep 8 13:59:12 2022] DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments/781bf9e8-460d-434e-8629-f5fd94f42731 => generated 0 bytes in 318 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Sep 08 13:59:12.548655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-edb2ff02-a958-40a9-b7f1-0b58d3b04ca3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:12.551122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-edb2ff02-a958-40a9-b7f1-0b58d3b04ca3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:12.557447 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-edb2ff02-a958-40a9-b7f1-0b58d3b04ca3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:12.558012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-edb2ff02-a958-40a9-b7f1-0b58d3b04ca3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:12.558484 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-edb2ff02-a958-40a9-b7f1-0b58d3b04ca3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:12.559218 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-edb2ff02-a958-40a9-b7f1-0b58d3b04ca3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:12.559643 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-edb2ff02-a958-40a9-b7f1-0b58d3b04ca3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:12.560064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-edb2ff02-a958-40a9-b7f1-0b58d3b04ca3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:12.583426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-49e619f4-92b0-40b0-aa03-4b58d1ab2e93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:12.591895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-edb2ff02-a958-40a9-b7f1-0b58d3b04ca3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047937 Sep 08 13:59:12.592643 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 307/735] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:12 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:59:12.595547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-49e619f4-92b0-40b0-aa03-4b58d1ab2e93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.166519 Sep 08 13:59:12.596256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 429/736] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:12 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:59:12.882348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-480b9176-6343-4831-82ca-dd79b6e39e89 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:12.882954 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-480b9176-6343-4831-82ca-dd79b6e39e89 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:12.888890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-480b9176-6343-4831-82ca-dd79b6e39e89 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:12.889371 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-480b9176-6343-4831-82ca-dd79b6e39e89 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:12.890469 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-480b9176-6343-4831-82ca-dd79b6e39e89 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:12.891164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-480b9176-6343-4831-82ca-dd79b6e39e89 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:12.891615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-480b9176-6343-4831-82ca-dd79b6e39e89 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:12.892039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-480b9176-6343-4831-82ca-dd79b6e39e89 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:13.022636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-480b9176-6343-4831-82ca-dd79b6e39e89 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:13.030204 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-480b9176-6343-4831-82ca-dd79b6e39e89 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.151723 Sep 08 13:59:13.030859 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 308/737] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:12 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:13.291577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1635fbf3-75cc-4fa6-8bd3-b6845231ea6c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:13.292179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1635fbf3-75cc-4fa6-8bd3-b6845231ea6c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:13.299214 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1635fbf3-75cc-4fa6-8bd3-b6845231ea6c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:13.299700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1635fbf3-75cc-4fa6-8bd3-b6845231ea6c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:13.300136 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1635fbf3-75cc-4fa6-8bd3-b6845231ea6c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:13.300827 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1635fbf3-75cc-4fa6-8bd3-b6845231ea6c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:13.301279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1635fbf3-75cc-4fa6-8bd3-b6845231ea6c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:13.301704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1635fbf3-75cc-4fa6-8bd3-b6845231ea6c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:13.430521 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1635fbf3-75cc-4fa6-8bd3-b6845231ea6c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:13.438075 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1635fbf3-75cc-4fa6-8bd3-b6845231ea6c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.150354 Sep 08 13:59:13.438649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 430/738] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:13 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:13.605281 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-16ecd144-05fa-4412-ad3c-3b98056672f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:13.607583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-16ecd144-05fa-4412-ad3c-3b98056672f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:13.614240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-16ecd144-05fa-4412-ad3c-3b98056672f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:13.614649 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-16ecd144-05fa-4412-ad3c-3b98056672f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:13.615077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-16ecd144-05fa-4412-ad3c-3b98056672f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:13.615759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-16ecd144-05fa-4412-ad3c-3b98056672f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:13.616203 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-16ecd144-05fa-4412-ad3c-3b98056672f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:13.616616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-16ecd144-05fa-4412-ad3c-3b98056672f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:13.619336 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c3366ad2-a523-4040-adc4-8f4014838ba0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:13.619976 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c3366ad2-a523-4040-adc4-8f4014838ba0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:13.626491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3366ad2-a523-4040-adc4-8f4014838ba0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:13.626984 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3366ad2-a523-4040-adc4-8f4014838ba0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:13.627449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3366ad2-a523-4040-adc4-8f4014838ba0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:13.628150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3366ad2-a523-4040-adc4-8f4014838ba0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:13.628607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3366ad2-a523-4040-adc4-8f4014838ba0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:13.629037 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3366ad2-a523-4040-adc4-8f4014838ba0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:13.648802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-16ecd144-05fa-4412-ad3c-3b98056672f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047507 Sep 08 13:59:13.649545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 309/739] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:13 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:13.761278 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c3366ad2-a523-4040-adc4-8f4014838ba0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:13.770091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c3366ad2-a523-4040-adc4-8f4014838ba0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.154528 Sep 08 13:59:13.770889 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 431/740] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:13 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:14.053564 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c4ac2009-ece6-4180-9fed-75ca30579ef6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:14.054075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c4ac2009-ece6-4180-9fed-75ca30579ef6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:14.060290 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c4ac2009-ece6-4180-9fed-75ca30579ef6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:14.060770 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c4ac2009-ece6-4180-9fed-75ca30579ef6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:14.061204 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c4ac2009-ece6-4180-9fed-75ca30579ef6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:14.062543 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c4ac2009-ece6-4180-9fed-75ca30579ef6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:14.063001 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c4ac2009-ece6-4180-9fed-75ca30579ef6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:14.063425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c4ac2009-ece6-4180-9fed-75ca30579ef6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:14.197929 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c4ac2009-ece6-4180-9fed-75ca30579ef6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:14.206013 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c4ac2009-ece6-4180-9fed-75ca30579ef6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.156514 Sep 08 13:59:14.206627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 310/741] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:14 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:14.462308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e39641f2-4b4c-4bfc-816a-bab7e70a5906 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:14.462910 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e39641f2-4b4c-4bfc-816a-bab7e70a5906 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:14.469892 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e39641f2-4b4c-4bfc-816a-bab7e70a5906 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:14.470413 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e39641f2-4b4c-4bfc-816a-bab7e70a5906 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:14.470853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e39641f2-4b4c-4bfc-816a-bab7e70a5906 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:14.471548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e39641f2-4b4c-4bfc-816a-bab7e70a5906 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:14.472006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e39641f2-4b4c-4bfc-816a-bab7e70a5906 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:14.472434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e39641f2-4b4c-4bfc-816a-bab7e70a5906 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:14.608095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e39641f2-4b4c-4bfc-816a-bab7e70a5906 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:14.616728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e39641f2-4b4c-4bfc-816a-bab7e70a5906 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.158376 Sep 08 13:59:14.617299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 432/742] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:14 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:14.662242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1048d7c4-5d4e-44d6-9c51-59c42957e605 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:14.664388 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1048d7c4-5d4e-44d6-9c51-59c42957e605 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:14.671074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1048d7c4-5d4e-44d6-9c51-59c42957e605 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:14.671560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1048d7c4-5d4e-44d6-9c51-59c42957e605 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:14.671994 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1048d7c4-5d4e-44d6-9c51-59c42957e605 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:14.672680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1048d7c4-5d4e-44d6-9c51-59c42957e605 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:14.673132 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1048d7c4-5d4e-44d6-9c51-59c42957e605 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:14.673554 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1048d7c4-5d4e-44d6-9c51-59c42957e605 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:14.703395 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1048d7c4-5d4e-44d6-9c51-59c42957e605 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045398 Sep 08 13:59:14.703952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 311/743] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:14 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:14.792934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-33a01966-6388-429e-ba67-c254c812debb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:14.793455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-33a01966-6388-429e-ba67-c254c812debb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:14.799470 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-33a01966-6388-429e-ba67-c254c812debb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:14.799948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-33a01966-6388-429e-ba67-c254c812debb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:14.800378 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-33a01966-6388-429e-ba67-c254c812debb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:14.801055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-33a01966-6388-429e-ba67-c254c812debb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:14.801503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-33a01966-6388-429e-ba67-c254c812debb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:14.801919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-33a01966-6388-429e-ba67-c254c812debb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:14.933150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-33a01966-6388-429e-ba67-c254c812debb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:14.941317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-33a01966-6388-429e-ba67-c254c812debb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.152342 Sep 08 13:59:14.941874 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 433/744] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:14 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:15.228215 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-72cd36a5-16cb-4f72-94dd-2ac3c3a1acd1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:15.228816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-72cd36a5-16cb-4f72-94dd-2ac3c3a1acd1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:15.234913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-72cd36a5-16cb-4f72-94dd-2ac3c3a1acd1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:15.235393 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-72cd36a5-16cb-4f72-94dd-2ac3c3a1acd1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:15.235826 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-72cd36a5-16cb-4f72-94dd-2ac3c3a1acd1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:15.236512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-72cd36a5-16cb-4f72-94dd-2ac3c3a1acd1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:15.236957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-72cd36a5-16cb-4f72-94dd-2ac3c3a1acd1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:15.237379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-72cd36a5-16cb-4f72-94dd-2ac3c3a1acd1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:15.369080 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-72cd36a5-16cb-4f72-94dd-2ac3c3a1acd1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:15.377499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-72cd36a5-16cb-4f72-94dd-2ac3c3a1acd1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.153265 Sep 08 13:59:15.378083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 312/745] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:15 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:15.640124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7fd17002-29b0-49df-b0c1-f406d5e80019 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:15.640735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7fd17002-29b0-49df-b0c1-f406d5e80019 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:15.647204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7fd17002-29b0-49df-b0c1-f406d5e80019 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:15.647691 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7fd17002-29b0-49df-b0c1-f406d5e80019 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:15.648157 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7fd17002-29b0-49df-b0c1-f406d5e80019 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:15.648850 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7fd17002-29b0-49df-b0c1-f406d5e80019 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:15.649313 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7fd17002-29b0-49df-b0c1-f406d5e80019 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:15.649739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7fd17002-29b0-49df-b0c1-f406d5e80019 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:15.716155 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a2d84c30-e62b-4635-8829-40faf01f6d50 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:15.718349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a2d84c30-e62b-4635-8829-40faf01f6d50 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:15.725301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a2d84c30-e62b-4635-8829-40faf01f6d50 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:15.725787 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a2d84c30-e62b-4635-8829-40faf01f6d50 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:15.726241 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a2d84c30-e62b-4635-8829-40faf01f6d50 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:15.726934 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a2d84c30-e62b-4635-8829-40faf01f6d50 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:15.727619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a2d84c30-e62b-4635-8829-40faf01f6d50 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:15.727789 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a2d84c30-e62b-4635-8829-40faf01f6d50 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:15.758407 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a2d84c30-e62b-4635-8829-40faf01f6d50 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046274 Sep 08 13:59:15.758985 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 313/746] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:15 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:59:15.779992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7fd17002-29b0-49df-b0c1-f406d5e80019 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:15.788404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7fd17002-29b0-49df-b0c1-f406d5e80019 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.152162 Sep 08 13:59:15.788966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 434/747] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:15 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:15.965055 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0e65e424-be43-4006-8864-5dc87394e298 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:15.965665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0e65e424-be43-4006-8864-5dc87394e298 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:15.972190 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e65e424-be43-4006-8864-5dc87394e298 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:15.972673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e65e424-be43-4006-8864-5dc87394e298 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:15.973110 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e65e424-be43-4006-8864-5dc87394e298 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:15.973792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e65e424-be43-4006-8864-5dc87394e298 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:15.974280 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e65e424-be43-4006-8864-5dc87394e298 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:15.974704 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e65e424-be43-4006-8864-5dc87394e298 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:15.978885 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:16.108035 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0e65e424-be43-4006-8864-5dc87394e298 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:16.117361 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0e65e424-be43-4006-8864-5dc87394e298 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.156199 Sep 08 13:59:16.117922 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 314/748] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:15 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:16.400435 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-02503c9f-e369-41e2-b122-af0f2c0bc378 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:16.412685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-02503c9f-e369-41e2-b122-af0f2c0bc378 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:16.412685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02503c9f-e369-41e2-b122-af0f2c0bc378 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:16.412685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02503c9f-e369-41e2-b122-af0f2c0bc378 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:16.412685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02503c9f-e369-41e2-b122-af0f2c0bc378 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:16.413201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02503c9f-e369-41e2-b122-af0f2c0bc378 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:16.413201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02503c9f-e369-41e2-b122-af0f2c0bc378 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:16.413201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02503c9f-e369-41e2-b122-af0f2c0bc378 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:16.555633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-02503c9f-e369-41e2-b122-af0f2c0bc378 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:16.563590 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-02503c9f-e369-41e2-b122-af0f2c0bc378 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.166982 Sep 08 13:59:16.564201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 435/749] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:16 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:16.771100 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a1aab83a-4b85-4c2b-aefc-984bf7a362a0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:16.773205 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a1aab83a-4b85-4c2b-aefc-984bf7a362a0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:16.779618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1aab83a-4b85-4c2b-aefc-984bf7a362a0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:16.780093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1aab83a-4b85-4c2b-aefc-984bf7a362a0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:16.780528 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1aab83a-4b85-4c2b-aefc-984bf7a362a0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:16.781213 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1aab83a-4b85-4c2b-aefc-984bf7a362a0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:16.781661 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1aab83a-4b85-4c2b-aefc-984bf7a362a0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:16.782078 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1aab83a-4b85-4c2b-aefc-984bf7a362a0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:16.811825 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ebd8b083-3e84-443c-b0dd-bdc2f7982118 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:16.812437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ebd8b083-3e84-443c-b0dd-bdc2f7982118 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:16.812796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a1aab83a-4b85-4c2b-aefc-984bf7a362a0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045747 Sep 08 13:59:16.813358 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 315/750] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:16 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:16.819024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ebd8b083-3e84-443c-b0dd-bdc2f7982118 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:16.819521 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ebd8b083-3e84-443c-b0dd-bdc2f7982118 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:16.819962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ebd8b083-3e84-443c-b0dd-bdc2f7982118 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:16.820663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ebd8b083-3e84-443c-b0dd-bdc2f7982118 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:16.821121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ebd8b083-3e84-443c-b0dd-bdc2f7982118 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:16.821551 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ebd8b083-3e84-443c-b0dd-bdc2f7982118 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:16.954086 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ebd8b083-3e84-443c-b0dd-bdc2f7982118 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:16.962447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ebd8b083-3e84-443c-b0dd-bdc2f7982118 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.154404 Sep 08 13:59:16.963017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 436/751] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:16 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:59:17.142086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-71d4434e-fd6a-48ca-a444-5a7532748f7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:17.143413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-71d4434e-fd6a-48ca-a444-5a7532748f7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:17.149380 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71d4434e-fd6a-48ca-a444-5a7532748f7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:17.149873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71d4434e-fd6a-48ca-a444-5a7532748f7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:17.150334 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71d4434e-fd6a-48ca-a444-5a7532748f7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:17.151025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71d4434e-fd6a-48ca-a444-5a7532748f7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:17.151479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71d4434e-fd6a-48ca-a444-5a7532748f7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:17.151902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71d4434e-fd6a-48ca-a444-5a7532748f7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:17.286014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-71d4434e-fd6a-48ca-a444-5a7532748f7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:17.294057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-71d4434e-fd6a-48ca-a444-5a7532748f7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.156016 Sep 08 13:59:17.294723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 316/752] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:17 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:17.587134 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5aff5050-dcc4-4198-86e0-afbf8cf07056 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:17.587812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5aff5050-dcc4-4198-86e0-afbf8cf07056 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:17.594079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aff5050-dcc4-4198-86e0-afbf8cf07056 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:17.594622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aff5050-dcc4-4198-86e0-afbf8cf07056 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:17.595065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aff5050-dcc4-4198-86e0-afbf8cf07056 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:17.595830 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aff5050-dcc4-4198-86e0-afbf8cf07056 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:17.596350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aff5050-dcc4-4198-86e0-afbf8cf07056 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:17.596779 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aff5050-dcc4-4198-86e0-afbf8cf07056 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:17.728275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5aff5050-dcc4-4198-86e0-afbf8cf07056 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:17.736571 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5aff5050-dcc4-4198-86e0-afbf8cf07056 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.153149 Sep 08 13:59:17.737142 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 437/753] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:17 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:17.825073 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-579572ba-c727-4d15-bba1-868e8a6f17d5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:17.827255 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-579572ba-c727-4d15-bba1-868e8a6f17d5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:17.834366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-579572ba-c727-4d15-bba1-868e8a6f17d5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:17.834856 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-579572ba-c727-4d15-bba1-868e8a6f17d5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:17.835292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-579572ba-c727-4d15-bba1-868e8a6f17d5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:17.835983 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-579572ba-c727-4d15-bba1-868e8a6f17d5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:17.836436 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-579572ba-c727-4d15-bba1-868e8a6f17d5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:17.836865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-579572ba-c727-4d15-bba1-868e8a6f17d5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:17.867415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-579572ba-c727-4d15-bba1-868e8a6f17d5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046294 Sep 08 13:59:17.867895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 317/754] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:17 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:17.986559 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ae760a35-37b7-41a0-b469-caef7177d5f3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:17.987150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ae760a35-37b7-41a0-b469-caef7177d5f3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:17.993564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae760a35-37b7-41a0-b469-caef7177d5f3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:17.994046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae760a35-37b7-41a0-b469-caef7177d5f3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:17.994510 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae760a35-37b7-41a0-b469-caef7177d5f3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:17.995199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae760a35-37b7-41a0-b469-caef7177d5f3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:17.995646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae760a35-37b7-41a0-b469-caef7177d5f3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:17.996064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae760a35-37b7-41a0-b469-caef7177d5f3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:18.128764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ae760a35-37b7-41a0-b469-caef7177d5f3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:18.136492 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ae760a35-37b7-41a0-b469-caef7177d5f3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.154430 Sep 08 13:59:18.137054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 438/755] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:17 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:18.316898 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3fbf2332-755e-47b8-b641-9e98660f41c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:18.317510 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3fbf2332-755e-47b8-b641-9e98660f41c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:18.323532 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3fbf2332-755e-47b8-b641-9e98660f41c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:18.325432 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3fbf2332-755e-47b8-b641-9e98660f41c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:18.325432 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3fbf2332-755e-47b8-b641-9e98660f41c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:18.325432 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3fbf2332-755e-47b8-b641-9e98660f41c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:18.325816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3fbf2332-755e-47b8-b641-9e98660f41c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:18.325981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3fbf2332-755e-47b8-b641-9e98660f41c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:18.456841 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3fbf2332-755e-47b8-b641-9e98660f41c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:18.465114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3fbf2332-755e-47b8-b641-9e98660f41c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.152040 Sep 08 13:59:18.465687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 318/756] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:18 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:18.759681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-af1c56e2-b6e9-401a-a893-492e45c20f10 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:18.760213 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-af1c56e2-b6e9-401a-a893-492e45c20f10 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:18.766245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af1c56e2-b6e9-401a-a893-492e45c20f10 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:18.766946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af1c56e2-b6e9-401a-a893-492e45c20f10 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:18.771294 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af1c56e2-b6e9-401a-a893-492e45c20f10 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:18.771294 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af1c56e2-b6e9-401a-a893-492e45c20f10 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:18.771294 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af1c56e2-b6e9-401a-a893-492e45c20f10 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:18.771294 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af1c56e2-b6e9-401a-a893-492e45c20f10 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:18.879586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e349839a-ee6b-4db6-9c97-207f849eb54d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:18.881807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e349839a-ee6b-4db6-9c97-207f849eb54d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:18.887984 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e349839a-ee6b-4db6-9c97-207f849eb54d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:18.888472 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e349839a-ee6b-4db6-9c97-207f849eb54d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:18.888905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e349839a-ee6b-4db6-9c97-207f849eb54d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:18.889606 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e349839a-ee6b-4db6-9c97-207f849eb54d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:18.890057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e349839a-ee6b-4db6-9c97-207f849eb54d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:18.890512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e349839a-ee6b-4db6-9c97-207f849eb54d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:18.903652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-af1c56e2-b6e9-401a-a893-492e45c20f10 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:18.911523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-af1c56e2-b6e9-401a-a893-492e45c20f10 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.155782 Sep 08 13:59:18.912090 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 439/757] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:18 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:18.921844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e349839a-ee6b-4db6-9c97-207f849eb54d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046796 Sep 08 13:59:18.922429 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 319/758] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:18 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:59:19.161849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f4e851a5-9729-45b6-a16e-d1a8dcc1afef tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:19.162472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f4e851a5-9729-45b6-a16e-d1a8dcc1afef tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:19.169083 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f4e851a5-9729-45b6-a16e-d1a8dcc1afef tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:19.169573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f4e851a5-9729-45b6-a16e-d1a8dcc1afef tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:19.170050 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f4e851a5-9729-45b6-a16e-d1a8dcc1afef tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:19.170798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f4e851a5-9729-45b6-a16e-d1a8dcc1afef tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:19.171255 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f4e851a5-9729-45b6-a16e-d1a8dcc1afef tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:19.171711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f4e851a5-9729-45b6-a16e-d1a8dcc1afef tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:19.175384 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:19.302875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f4e851a5-9729-45b6-a16e-d1a8dcc1afef tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:19.310553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f4e851a5-9729-45b6-a16e-d1a8dcc1afef tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.154015 Sep 08 13:59:19.311124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 440/759] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:19 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:19.488702 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c4d6e8d7-5a0c-46ce-b891-81b61db5cd34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:19.489300 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c4d6e8d7-5a0c-46ce-b891-81b61db5cd34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:19.495797 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c4d6e8d7-5a0c-46ce-b891-81b61db5cd34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:19.496278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c4d6e8d7-5a0c-46ce-b891-81b61db5cd34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:19.496705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c4d6e8d7-5a0c-46ce-b891-81b61db5cd34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:19.497393 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c4d6e8d7-5a0c-46ce-b891-81b61db5cd34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:19.497838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c4d6e8d7-5a0c-46ce-b891-81b61db5cd34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:19.498280 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c4d6e8d7-5a0c-46ce-b891-81b61db5cd34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:19.632508 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c4d6e8d7-5a0c-46ce-b891-81b61db5cd34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:19.641393 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c4d6e8d7-5a0c-46ce-b891-81b61db5cd34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.157071 Sep 08 13:59:19.642565 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 320/760] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:19 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:19.934378 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3269a629-e024-4fea-8cf8-cf06041836e9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:19.934675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6582eb5e-da16-478b-871b-3036d37d788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:19.935141 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3269a629-e024-4fea-8cf8-cf06041836e9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:19.936790 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6582eb5e-da16-478b-871b-3036d37d788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:19.942045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3269a629-e024-4fea-8cf8-cf06041836e9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:19.942557 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3269a629-e024-4fea-8cf8-cf06041836e9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:19.942991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3269a629-e024-4fea-8cf8-cf06041836e9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:19.943689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3269a629-e024-4fea-8cf8-cf06041836e9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:19.944004 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6582eb5e-da16-478b-871b-3036d37d788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:19.944144 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3269a629-e024-4fea-8cf8-cf06041836e9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:19.944493 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6582eb5e-da16-478b-871b-3036d37d788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:19.944592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3269a629-e024-4fea-8cf8-cf06041836e9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:19.944930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6582eb5e-da16-478b-871b-3036d37d788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:19.945629 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6582eb5e-da16-478b-871b-3036d37d788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:19.946080 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6582eb5e-da16-478b-871b-3036d37d788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:19.946535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6582eb5e-da16-478b-871b-3036d37d788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:19.975905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6582eb5e-da16-478b-871b-3036d37d788e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045327 Sep 08 13:59:19.976776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 321/761] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:19 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:20.075318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3269a629-e024-4fea-8cf8-cf06041836e9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:20.083643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3269a629-e024-4fea-8cf8-cf06041836e9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.153134 Sep 08 13:59:20.084208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 441/762] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:19 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:20.334449 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d3c493d7-0dad-4e2c-aadf-cdeb197b98ea tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:20.335059 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d3c493d7-0dad-4e2c-aadf-cdeb197b98ea tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:20.341106 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d3c493d7-0dad-4e2c-aadf-cdeb197b98ea tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:20.341589 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d3c493d7-0dad-4e2c-aadf-cdeb197b98ea tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:20.342022 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d3c493d7-0dad-4e2c-aadf-cdeb197b98ea tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:20.342729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d3c493d7-0dad-4e2c-aadf-cdeb197b98ea tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:20.343182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d3c493d7-0dad-4e2c-aadf-cdeb197b98ea tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:20.343606 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d3c493d7-0dad-4e2c-aadf-cdeb197b98ea tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:20.473782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d3c493d7-0dad-4e2c-aadf-cdeb197b98ea tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:20.481548 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d3c493d7-0dad-4e2c-aadf-cdeb197b98ea tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.150988 Sep 08 13:59:20.482122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 322/763] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:20 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:20.665135 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4dc20f9a-c6ae-40a3-aeb2-d118fb4f1b96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:20.665739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4dc20f9a-c6ae-40a3-aeb2-d118fb4f1b96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:20.672216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4dc20f9a-c6ae-40a3-aeb2-d118fb4f1b96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:20.673216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4dc20f9a-c6ae-40a3-aeb2-d118fb4f1b96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:20.674031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4dc20f9a-c6ae-40a3-aeb2-d118fb4f1b96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:20.674540 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4dc20f9a-c6ae-40a3-aeb2-d118fb4f1b96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:20.674999 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4dc20f9a-c6ae-40a3-aeb2-d118fb4f1b96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:20.675424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4dc20f9a-c6ae-40a3-aeb2-d118fb4f1b96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:20.807181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4dc20f9a-c6ae-40a3-aeb2-d118fb4f1b96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:20.817868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4dc20f9a-c6ae-40a3-aeb2-d118fb4f1b96 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.156547 Sep 08 13:59:20.818490 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 442/764] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:20 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:20.989441 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2a666799-164c-40a1-9171-049511d44c0c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:20.991612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2a666799-164c-40a1-9171-049511d44c0c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:20.998037 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2a666799-164c-40a1-9171-049511d44c0c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:20.998549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2a666799-164c-40a1-9171-049511d44c0c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:20.998978 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2a666799-164c-40a1-9171-049511d44c0c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:20.999683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2a666799-164c-40a1-9171-049511d44c0c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:21.000113 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2a666799-164c-40a1-9171-049511d44c0c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:21.000534 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2a666799-164c-40a1-9171-049511d44c0c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:21.029615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2a666799-164c-40a1-9171-049511d44c0c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044888 Sep 08 13:59:21.030181 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 323/765] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:20 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:21.106866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c3224cb0-17fb-4428-ab6e-0d58549e771d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:21.107475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c3224cb0-17fb-4428-ab6e-0d58549e771d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:21.114168 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3224cb0-17fb-4428-ab6e-0d58549e771d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:21.114679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3224cb0-17fb-4428-ab6e-0d58549e771d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:21.115121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3224cb0-17fb-4428-ab6e-0d58549e771d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:21.115813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3224cb0-17fb-4428-ab6e-0d58549e771d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:21.116270 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3224cb0-17fb-4428-ab6e-0d58549e771d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:21.116698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3224cb0-17fb-4428-ab6e-0d58549e771d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:21.256221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c3224cb0-17fb-4428-ab6e-0d58549e771d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:21.263838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c3224cb0-17fb-4428-ab6e-0d58549e771d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.160810 Sep 08 13:59:21.264400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 443/766] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:21 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:21.505036 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d40a7928-e0bc-4b99-afab-283b78ffffc8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:21.505557 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d40a7928-e0bc-4b99-afab-283b78ffffc8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:21.507469 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=91 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:21.512140 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d40a7928-e0bc-4b99-afab-283b78ffffc8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:21.512617 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d40a7928-e0bc-4b99-afab-283b78ffffc8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:21.513046 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d40a7928-e0bc-4b99-afab-283b78ffffc8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:21.518602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d40a7928-e0bc-4b99-afab-283b78ffffc8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:21.518602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d40a7928-e0bc-4b99-afab-283b78ffffc8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:21.518602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d40a7928-e0bc-4b99-afab-283b78ffffc8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:21.659493 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d40a7928-e0bc-4b99-afab-283b78ffffc8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:21.662633 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d40a7928-e0bc-4b99-afab-283b78ffffc8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.161564 Sep 08 13:59:21.663219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 324/767] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:21 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:21.841768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-feaaebd4-d870-43bb-98e6-4475313b43cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:21.842392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-feaaebd4-d870-43bb-98e6-4475313b43cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:21.848930 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-feaaebd4-d870-43bb-98e6-4475313b43cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:21.849411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-feaaebd4-d870-43bb-98e6-4475313b43cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:21.850092 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-feaaebd4-d870-43bb-98e6-4475313b43cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:21.850547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-feaaebd4-d870-43bb-98e6-4475313b43cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:21.850992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-feaaebd4-d870-43bb-98e6-4475313b43cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:21.851408 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-feaaebd4-d870-43bb-98e6-4475313b43cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:21.980844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-feaaebd4-d870-43bb-98e6-4475313b43cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:21.989045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-feaaebd4-d870-43bb-98e6-4475313b43cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.151172 Sep 08 13:59:21.989608 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 444/768] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:21 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:22.043116 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b3093cfc-2817-49a4-86df-1a395279f7af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:22.045173 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b3093cfc-2817-49a4-86df-1a395279f7af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:22.052303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3093cfc-2817-49a4-86df-1a395279f7af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:22.052784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3093cfc-2817-49a4-86df-1a395279f7af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:22.053216 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3093cfc-2817-49a4-86df-1a395279f7af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:22.053903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3093cfc-2817-49a4-86df-1a395279f7af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:22.054383 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3093cfc-2817-49a4-86df-1a395279f7af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:22.054804 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3093cfc-2817-49a4-86df-1a395279f7af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:22.084699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b3093cfc-2817-49a4-86df-1a395279f7af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045676 Sep 08 13:59:22.085251 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 325/769] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:22 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:59:22.287128 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ead25ef0-13e0-4f7c-b86a-b9b4d3a6eb95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:22.287737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ead25ef0-13e0-4f7c-b86a-b9b4d3a6eb95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:22.293688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ead25ef0-13e0-4f7c-b86a-b9b4d3a6eb95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:22.294110 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ead25ef0-13e0-4f7c-b86a-b9b4d3a6eb95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:22.294585 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ead25ef0-13e0-4f7c-b86a-b9b4d3a6eb95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:22.295375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ead25ef0-13e0-4f7c-b86a-b9b4d3a6eb95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:22.295835 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ead25ef0-13e0-4f7c-b86a-b9b4d3a6eb95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:22.296263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ead25ef0-13e0-4f7c-b86a-b9b4d3a6eb95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:22.429456 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ead25ef0-13e0-4f7c-b86a-b9b4d3a6eb95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:22.437221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ead25ef0-13e0-4f7c-b86a-b9b4d3a6eb95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.154451 Sep 08 13:59:22.438414 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 445/770] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:22 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:22.685295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-46cb772f-08cd-4bad-9695-1a68514d315b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:22.685893 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-46cb772f-08cd-4bad-9695-1a68514d315b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:22.692955 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-46cb772f-08cd-4bad-9695-1a68514d315b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:22.693539 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-46cb772f-08cd-4bad-9695-1a68514d315b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:22.694216 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-46cb772f-08cd-4bad-9695-1a68514d315b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:22.695355 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-46cb772f-08cd-4bad-9695-1a68514d315b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:22.696040 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-46cb772f-08cd-4bad-9695-1a68514d315b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:22.696434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-46cb772f-08cd-4bad-9695-1a68514d315b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:22.833118 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-46cb772f-08cd-4bad-9695-1a68514d315b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:22.840922 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-46cb772f-08cd-4bad-9695-1a68514d315b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.159493 Sep 08 13:59:22.841475 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 326/771] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:22 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:23.012307 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2ac1566c-e73c-4ff7-a305-696ab1faa88a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:23.012859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2ac1566c-e73c-4ff7-a305-696ab1faa88a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:23.015130 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=86 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:23.020067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ac1566c-e73c-4ff7-a305-696ab1faa88a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:23.020617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ac1566c-e73c-4ff7-a305-696ab1faa88a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:23.021050 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ac1566c-e73c-4ff7-a305-696ab1faa88a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:23.026985 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ac1566c-e73c-4ff7-a305-696ab1faa88a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:23.026985 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ac1566c-e73c-4ff7-a305-696ab1faa88a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:23.026985 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ac1566c-e73c-4ff7-a305-696ab1faa88a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:23.097048 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-00d1c42a-f264-4d9a-9a7c-02b011531975 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:23.099233 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-00d1c42a-f264-4d9a-9a7c-02b011531975 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:23.106023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00d1c42a-f264-4d9a-9a7c-02b011531975 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:23.106539 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00d1c42a-f264-4d9a-9a7c-02b011531975 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:23.107600 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00d1c42a-f264-4d9a-9a7c-02b011531975 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:23.108302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00d1c42a-f264-4d9a-9a7c-02b011531975 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:23.108749 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00d1c42a-f264-4d9a-9a7c-02b011531975 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:23.109662 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00d1c42a-f264-4d9a-9a7c-02b011531975 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:23.141147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-00d1c42a-f264-4d9a-9a7c-02b011531975 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048038 Sep 08 13:59:23.141964 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 327/772] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:23 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:59:23.159947 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2ac1566c-e73c-4ff7-a305-696ab1faa88a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:23.168149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2ac1566c-e73c-4ff7-a305-696ab1faa88a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.159907 Sep 08 13:59:23.168723 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 446/773] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:23 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:23.459690 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9e50a84d-ed36-4a52-8d81-92f2ab86c417 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:23.460289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9e50a84d-ed36-4a52-8d81-92f2ab86c417 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:23.466365 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e50a84d-ed36-4a52-8d81-92f2ab86c417 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:23.466911 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e50a84d-ed36-4a52-8d81-92f2ab86c417 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:23.467349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e50a84d-ed36-4a52-8d81-92f2ab86c417 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:23.468070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e50a84d-ed36-4a52-8d81-92f2ab86c417 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:23.469030 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e50a84d-ed36-4a52-8d81-92f2ab86c417 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:23.469447 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e50a84d-ed36-4a52-8d81-92f2ab86c417 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:23.609454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9e50a84d-ed36-4a52-8d81-92f2ab86c417 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:23.617362 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9e50a84d-ed36-4a52-8d81-92f2ab86c417 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.161567 Sep 08 13:59:23.617932 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 328/774] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:23 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:23.865648 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-01ac624b-1ba7-4c18-9c12-fefbfa979cda tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:23.866297 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-01ac624b-1ba7-4c18-9c12-fefbfa979cda tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:23.872349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01ac624b-1ba7-4c18-9c12-fefbfa979cda tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:23.872844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01ac624b-1ba7-4c18-9c12-fefbfa979cda tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:23.873264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01ac624b-1ba7-4c18-9c12-fefbfa979cda tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:23.873951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01ac624b-1ba7-4c18-9c12-fefbfa979cda tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:23.874439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01ac624b-1ba7-4c18-9c12-fefbfa979cda tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:23.875435 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01ac624b-1ba7-4c18-9c12-fefbfa979cda tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:24.005351 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-01ac624b-1ba7-4c18-9c12-fefbfa979cda tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:24.013103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-01ac624b-1ba7-4c18-9c12-fefbfa979cda tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.151345 Sep 08 13:59:24.013694 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 447/775] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:23 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:24.153905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-27c70448-0ac2-4ee8-bc41-28664fd55608 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:24.156568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-27c70448-0ac2-4ee8-bc41-28664fd55608 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:24.163355 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27c70448-0ac2-4ee8-bc41-28664fd55608 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:24.163831 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27c70448-0ac2-4ee8-bc41-28664fd55608 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:24.168777 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27c70448-0ac2-4ee8-bc41-28664fd55608 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:24.168777 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27c70448-0ac2-4ee8-bc41-28664fd55608 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:24.168777 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27c70448-0ac2-4ee8-bc41-28664fd55608 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:24.168777 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27c70448-0ac2-4ee8-bc41-28664fd55608 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:24.191860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5df30d78-bd1a-468d-bcb9-e8c9ee7aa9c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:24.192467 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5df30d78-bd1a-468d-bcb9-e8c9ee7aa9c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:24.196309 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-27c70448-0ac2-4ee8-bc41-28664fd55608 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046823 Sep 08 13:59:24.196889 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 329/776] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:24 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:24.198953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5df30d78-bd1a-468d-bcb9-e8c9ee7aa9c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:24.199454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5df30d78-bd1a-468d-bcb9-e8c9ee7aa9c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:24.200029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5df30d78-bd1a-468d-bcb9-e8c9ee7aa9c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:24.200658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5df30d78-bd1a-468d-bcb9-e8c9ee7aa9c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:24.201119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5df30d78-bd1a-468d-bcb9-e8c9ee7aa9c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:24.201550 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5df30d78-bd1a-468d-bcb9-e8c9ee7aa9c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:24.354439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5df30d78-bd1a-468d-bcb9-e8c9ee7aa9c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:24.362989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5df30d78-bd1a-468d-bcb9-e8c9ee7aa9c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.175224 Sep 08 13:59:24.369054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 448/777] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:24 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:59:24.641245 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8f9e603e-5af0-4c06-889e-a161fd4ea4c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:24.641367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8f9e603e-5af0-4c06-889e-a161fd4ea4c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:24.647606 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f9e603e-5af0-4c06-889e-a161fd4ea4c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:24.648087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f9e603e-5af0-4c06-889e-a161fd4ea4c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:24.648517 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f9e603e-5af0-4c06-889e-a161fd4ea4c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:24.649211 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f9e603e-5af0-4c06-889e-a161fd4ea4c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:24.649663 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f9e603e-5af0-4c06-889e-a161fd4ea4c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:24.650092 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f9e603e-5af0-4c06-889e-a161fd4ea4c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:24.787672 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8f9e603e-5af0-4c06-889e-a161fd4ea4c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:24.791295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:24.797018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8f9e603e-5af0-4c06-889e-a161fd4ea4c5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.159608 Sep 08 13:59:24.797367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 330/778] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:24 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:25.037321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5a1babce-25cf-4a1b-9983-16462f0f95f4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:25.037881 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5a1babce-25cf-4a1b-9983-16462f0f95f4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:25.044104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a1babce-25cf-4a1b-9983-16462f0f95f4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:25.044586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a1babce-25cf-4a1b-9983-16462f0f95f4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:25.045016 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a1babce-25cf-4a1b-9983-16462f0f95f4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:25.045701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a1babce-25cf-4a1b-9983-16462f0f95f4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:25.046828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a1babce-25cf-4a1b-9983-16462f0f95f4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:25.047806 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a1babce-25cf-4a1b-9983-16462f0f95f4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:25.174294 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5a1babce-25cf-4a1b-9983-16462f0f95f4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:25.181981 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5a1babce-25cf-4a1b-9983-16462f0f95f4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.148582 Sep 08 13:59:25.182572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 449/779] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:25 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:25.210032 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-577dca4e-246d-4688-a6d5-fbef52eb818c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:25.212200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-577dca4e-246d-4688-a6d5-fbef52eb818c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:25.219732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-577dca4e-246d-4688-a6d5-fbef52eb818c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:25.220316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-577dca4e-246d-4688-a6d5-fbef52eb818c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:25.220979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-577dca4e-246d-4688-a6d5-fbef52eb818c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:25.221714 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-577dca4e-246d-4688-a6d5-fbef52eb818c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:25.222252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-577dca4e-246d-4688-a6d5-fbef52eb818c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:25.222682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-577dca4e-246d-4688-a6d5-fbef52eb818c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:25.252591 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-577dca4e-246d-4688-a6d5-fbef52eb818c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046898 Sep 08 13:59:25.253169 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 331/780] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:25 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:25.386280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-27212789-3e14-4d06-9eed-964478028094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:25.386855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-27212789-3e14-4d06-9eed-964478028094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:25.393286 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27212789-3e14-4d06-9eed-964478028094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:25.393768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27212789-3e14-4d06-9eed-964478028094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:25.394227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27212789-3e14-4d06-9eed-964478028094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:25.394912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27212789-3e14-4d06-9eed-964478028094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:25.395365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27212789-3e14-4d06-9eed-964478028094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:25.395945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27212789-3e14-4d06-9eed-964478028094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:25.530625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-27212789-3e14-4d06-9eed-964478028094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:25.539869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-27212789-3e14-4d06-9eed-964478028094 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.157601 Sep 08 13:59:25.540442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 450/781] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:25 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:25.820369 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-01087227-0e1f-41e6-9da9-a173c4a7ca44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:25.821007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-01087227-0e1f-41e6-9da9-a173c4a7ca44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:25.827590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-01087227-0e1f-41e6-9da9-a173c4a7ca44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:25.827699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-01087227-0e1f-41e6-9da9-a173c4a7ca44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:25.828091 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-01087227-0e1f-41e6-9da9-a173c4a7ca44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:25.828785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-01087227-0e1f-41e6-9da9-a173c4a7ca44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:25.829238 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-01087227-0e1f-41e6-9da9-a173c4a7ca44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:25.829658 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-01087227-0e1f-41e6-9da9-a173c4a7ca44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:25.980365 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-01087227-0e1f-41e6-9da9-a173c4a7ca44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:25.988896 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-01087227-0e1f-41e6-9da9-a173c4a7ca44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.172844 Sep 08 13:59:25.989473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 332/782] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:25 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:26.205348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-43e379ba-8e25-405d-9ae7-b934b6914cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:26.205890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-43e379ba-8e25-405d-9ae7-b934b6914cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:26.212467 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43e379ba-8e25-405d-9ae7-b934b6914cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:26.212951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43e379ba-8e25-405d-9ae7-b934b6914cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:26.213385 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43e379ba-8e25-405d-9ae7-b934b6914cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:26.214077 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43e379ba-8e25-405d-9ae7-b934b6914cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:26.214563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43e379ba-8e25-405d-9ae7-b934b6914cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:26.214990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43e379ba-8e25-405d-9ae7-b934b6914cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:26.267502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d51e0578-d3f3-4dcf-b77d-361ec76e8e9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:26.271027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d51e0578-d3f3-4dcf-b77d-361ec76e8e9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:26.276052 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d51e0578-d3f3-4dcf-b77d-361ec76e8e9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:26.276619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d51e0578-d3f3-4dcf-b77d-361ec76e8e9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:26.277052 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d51e0578-d3f3-4dcf-b77d-361ec76e8e9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:26.277764 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d51e0578-d3f3-4dcf-b77d-361ec76e8e9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:26.278241 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d51e0578-d3f3-4dcf-b77d-361ec76e8e9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:26.278664 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d51e0578-d3f3-4dcf-b77d-361ec76e8e9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:26.315394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d51e0578-d3f3-4dcf-b77d-361ec76e8e9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053001 Sep 08 13:59:26.316063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 333/783] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:26 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:26.357505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-43e379ba-8e25-405d-9ae7-b934b6914cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:26.366058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-43e379ba-8e25-405d-9ae7-b934b6914cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.164681 Sep 08 13:59:26.366650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 451/784] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:26 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:26.569349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ceee1e5b-1035-47ef-98fd-c9da5734c512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:26.569886 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ceee1e5b-1035-47ef-98fd-c9da5734c512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:26.575853 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ceee1e5b-1035-47ef-98fd-c9da5734c512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:26.576337 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ceee1e5b-1035-47ef-98fd-c9da5734c512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:26.576789 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ceee1e5b-1035-47ef-98fd-c9da5734c512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:26.578168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ceee1e5b-1035-47ef-98fd-c9da5734c512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:26.578775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ceee1e5b-1035-47ef-98fd-c9da5734c512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:26.588136 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ceee1e5b-1035-47ef-98fd-c9da5734c512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:26.748440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ceee1e5b-1035-47ef-98fd-c9da5734c512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:26.757074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ceee1e5b-1035-47ef-98fd-c9da5734c512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.191955 Sep 08 13:59:26.757645 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 334/785] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:26 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:27.020583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4f74891b-505e-41a1-adfb-8c90d9d1c97b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:27.021441 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4f74891b-505e-41a1-adfb-8c90d9d1c97b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:27.034824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f74891b-505e-41a1-adfb-8c90d9d1c97b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:27.035310 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f74891b-505e-41a1-adfb-8c90d9d1c97b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:27.035745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f74891b-505e-41a1-adfb-8c90d9d1c97b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:27.036447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f74891b-505e-41a1-adfb-8c90d9d1c97b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:27.036901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f74891b-505e-41a1-adfb-8c90d9d1c97b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:27.037336 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f74891b-505e-41a1-adfb-8c90d9d1c97b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:27.174783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4f74891b-505e-41a1-adfb-8c90d9d1c97b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:27.182574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4f74891b-505e-41a1-adfb-8c90d9d1c97b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.174518 Sep 08 13:59:27.183144 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 452/786] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:27 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:27.328250 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7b889dfa-b521-4bb6-877e-67c923efafaa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:27.330528 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7b889dfa-b521-4bb6-877e-67c923efafaa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:27.337235 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b889dfa-b521-4bb6-877e-67c923efafaa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:27.337711 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b889dfa-b521-4bb6-877e-67c923efafaa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:27.338162 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b889dfa-b521-4bb6-877e-67c923efafaa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:27.338851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b889dfa-b521-4bb6-877e-67c923efafaa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:27.339388 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b889dfa-b521-4bb6-877e-67c923efafaa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:27.339871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b889dfa-b521-4bb6-877e-67c923efafaa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:27.370108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7b889dfa-b521-4bb6-877e-67c923efafaa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045881 Sep 08 13:59:27.370610 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 335/787] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:27 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:59:27.388736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-82a8847b-f890-49fe-9777-8066b4461c11 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:27.389347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-82a8847b-f890-49fe-9777-8066b4461c11 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:27.395925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-82a8847b-f890-49fe-9777-8066b4461c11 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:27.396415 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-82a8847b-f890-49fe-9777-8066b4461c11 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:27.396850 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-82a8847b-f890-49fe-9777-8066b4461c11 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:27.397548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-82a8847b-f890-49fe-9777-8066b4461c11 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:27.397999 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-82a8847b-f890-49fe-9777-8066b4461c11 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:27.398461 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-82a8847b-f890-49fe-9777-8066b4461c11 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:27.526871 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-82a8847b-f890-49fe-9777-8066b4461c11 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:27.534985 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-82a8847b-f890-49fe-9777-8066b4461c11 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.150126 Sep 08 13:59:27.535557 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 453/788] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:27 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:27.782017 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-792bac5a-5b00-4e56-8db0-5d48ead1c7cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:27.782584 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-792bac5a-5b00-4e56-8db0-5d48ead1c7cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:27.788691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-792bac5a-5b00-4e56-8db0-5d48ead1c7cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:27.789170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-792bac5a-5b00-4e56-8db0-5d48ead1c7cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:27.789693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-792bac5a-5b00-4e56-8db0-5d48ead1c7cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:27.790423 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-792bac5a-5b00-4e56-8db0-5d48ead1c7cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:27.790877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-792bac5a-5b00-4e56-8db0-5d48ead1c7cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:27.791302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-792bac5a-5b00-4e56-8db0-5d48ead1c7cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:27.925116 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-792bac5a-5b00-4e56-8db0-5d48ead1c7cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:27.934560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-792bac5a-5b00-4e56-8db0-5d48ead1c7cb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.157423 Sep 08 13:59:27.935137 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 336/789] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:27 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:28.207158 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c3f90ce7-15c8-4638-9f9c-f8ac32422211 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:28.207715 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c3f90ce7-15c8-4638-9f9c-f8ac32422211 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:28.213908 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3f90ce7-15c8-4638-9f9c-f8ac32422211 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:28.214444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3f90ce7-15c8-4638-9f9c-f8ac32422211 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:28.214887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3f90ce7-15c8-4638-9f9c-f8ac32422211 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:28.215594 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3f90ce7-15c8-4638-9f9c-f8ac32422211 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:28.216048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3f90ce7-15c8-4638-9f9c-f8ac32422211 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:28.216477 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3f90ce7-15c8-4638-9f9c-f8ac32422211 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:28.356691 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c3f90ce7-15c8-4638-9f9c-f8ac32422211 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:28.365174 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c3f90ce7-15c8-4638-9f9c-f8ac32422211 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.162773 Sep 08 13:59:28.365746 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 454/790] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:28 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:28.385100 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-93db922b-9fe5-4c9a-b78c-5e3ec0a3730e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:28.387396 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-93db922b-9fe5-4c9a-b78c-5e3ec0a3730e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:28.394050 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-93db922b-9fe5-4c9a-b78c-5e3ec0a3730e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:28.394564 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-93db922b-9fe5-4c9a-b78c-5e3ec0a3730e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:28.394999 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-93db922b-9fe5-4c9a-b78c-5e3ec0a3730e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:28.395688 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-93db922b-9fe5-4c9a-b78c-5e3ec0a3730e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:28.396139 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-93db922b-9fe5-4c9a-b78c-5e3ec0a3730e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:28.396563 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-93db922b-9fe5-4c9a-b78c-5e3ec0a3730e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:28.429320 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-93db922b-9fe5-4c9a-b78c-5e3ec0a3730e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047834 Sep 08 13:59:28.430080 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 337/791] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:28 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:28.558938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2b746d18-1d7e-4a53-b58a-e51a416b0cb7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:28.559569 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2b746d18-1d7e-4a53-b58a-e51a416b0cb7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:28.566678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b746d18-1d7e-4a53-b58a-e51a416b0cb7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:28.567206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b746d18-1d7e-4a53-b58a-e51a416b0cb7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:28.567638 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b746d18-1d7e-4a53-b58a-e51a416b0cb7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:28.568328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b746d18-1d7e-4a53-b58a-e51a416b0cb7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:28.568778 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b746d18-1d7e-4a53-b58a-e51a416b0cb7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:28.569197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b746d18-1d7e-4a53-b58a-e51a416b0cb7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:28.712971 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2b746d18-1d7e-4a53-b58a-e51a416b0cb7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:28.723617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2b746d18-1d7e-4a53-b58a-e51a416b0cb7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.168300 Sep 08 13:59:28.724208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 455/792] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:28 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:28.960567 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-dde169be-ad14-4ca1-9d3c-65a49cf1d048 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:28.961222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-dde169be-ad14-4ca1-9d3c-65a49cf1d048 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:28.969774 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dde169be-ad14-4ca1-9d3c-65a49cf1d048 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:28.970370 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dde169be-ad14-4ca1-9d3c-65a49cf1d048 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:28.970856 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dde169be-ad14-4ca1-9d3c-65a49cf1d048 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:28.971589 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dde169be-ad14-4ca1-9d3c-65a49cf1d048 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:28.972081 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dde169be-ad14-4ca1-9d3c-65a49cf1d048 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:28.972545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dde169be-ad14-4ca1-9d3c-65a49cf1d048 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:29.104775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-dde169be-ad14-4ca1-9d3c-65a49cf1d048 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:29.113145 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-dde169be-ad14-4ca1-9d3c-65a49cf1d048 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.156904 Sep 08 13:59:29.113799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 338/793] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:28 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:29.391397 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9acb7846-2670-4d5f-a0dd-d6c8e825df92 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:29.391999 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9acb7846-2670-4d5f-a0dd-d6c8e825df92 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:29.397982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9acb7846-2670-4d5f-a0dd-d6c8e825df92 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:29.398495 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9acb7846-2670-4d5f-a0dd-d6c8e825df92 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:29.398934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9acb7846-2670-4d5f-a0dd-d6c8e825df92 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:29.399616 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9acb7846-2670-4d5f-a0dd-d6c8e825df92 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:29.400066 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9acb7846-2670-4d5f-a0dd-d6c8e825df92 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:29.400501 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9acb7846-2670-4d5f-a0dd-d6c8e825df92 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:29.442529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2f97a18d-f66b-4333-a542-3fd6725d7be1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:29.444875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2f97a18d-f66b-4333-a542-3fd6725d7be1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:29.451306 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2f97a18d-f66b-4333-a542-3fd6725d7be1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:29.451788 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2f97a18d-f66b-4333-a542-3fd6725d7be1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:29.452746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2f97a18d-f66b-4333-a542-3fd6725d7be1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:29.453431 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2f97a18d-f66b-4333-a542-3fd6725d7be1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:29.453873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2f97a18d-f66b-4333-a542-3fd6725d7be1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:29.454326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2f97a18d-f66b-4333-a542-3fd6725d7be1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:29.484017 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2f97a18d-f66b-4333-a542-3fd6725d7be1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045547 Sep 08 13:59:29.484590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 339/794] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:29 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:29.532024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9acb7846-2670-4d5f-a0dd-d6c8e825df92 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:29.534264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:29.540315 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9acb7846-2670-4d5f-a0dd-d6c8e825df92 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.153440 Sep 08 13:59:29.540882 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 456/795] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:29 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:29.748815 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ce83df2b-c34b-4cfa-8b1d-046bede17490 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:29.749364 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ce83df2b-c34b-4cfa-8b1d-046bede17490 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:29.755371 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce83df2b-c34b-4cfa-8b1d-046bede17490 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:29.755851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce83df2b-c34b-4cfa-8b1d-046bede17490 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:29.756284 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce83df2b-c34b-4cfa-8b1d-046bede17490 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:29.756967 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce83df2b-c34b-4cfa-8b1d-046bede17490 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:29.757415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce83df2b-c34b-4cfa-8b1d-046bede17490 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:29.757837 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce83df2b-c34b-4cfa-8b1d-046bede17490 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:29.887879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ce83df2b-c34b-4cfa-8b1d-046bede17490 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:29.895746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ce83df2b-c34b-4cfa-8b1d-046bede17490 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.151378 Sep 08 13:59:29.896310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 340/796] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:29 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:30.137054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a02f4f29-d959-4a33-af78-d34a66ae19b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:30.137922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a02f4f29-d959-4a33-af78-d34a66ae19b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:30.147228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a02f4f29-d959-4a33-af78-d34a66ae19b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:30.147724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a02f4f29-d959-4a33-af78-d34a66ae19b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:30.148163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a02f4f29-d959-4a33-af78-d34a66ae19b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:30.148974 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a02f4f29-d959-4a33-af78-d34a66ae19b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:30.149671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a02f4f29-d959-4a33-af78-d34a66ae19b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:30.150367 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a02f4f29-d959-4a33-af78-d34a66ae19b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:30.289509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a02f4f29-d959-4a33-af78-d34a66ae19b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:30.298428 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a02f4f29-d959-4a33-af78-d34a66ae19b9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.165593 Sep 08 13:59:30.299050 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 457/797] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:30 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:30.497000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4b205c75-d323-4403-81cf-35b88bc00432 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:30.499207 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4b205c75-d323-4403-81cf-35b88bc00432 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:30.505693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b205c75-d323-4403-81cf-35b88bc00432 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:30.506210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b205c75-d323-4403-81cf-35b88bc00432 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:30.506703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b205c75-d323-4403-81cf-35b88bc00432 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:30.507394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b205c75-d323-4403-81cf-35b88bc00432 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:30.507847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b205c75-d323-4403-81cf-35b88bc00432 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:30.508274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b205c75-d323-4403-81cf-35b88bc00432 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:30.538631 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4b205c75-d323-4403-81cf-35b88bc00432 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045723 Sep 08 13:59:30.539185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 341/798] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:30 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:59:30.563167 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5a179ab6-c5e4-4d83-9cf5-9addf22f86b7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:30.563824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5a179ab6-c5e4-4d83-9cf5-9addf22f86b7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:30.569876 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a179ab6-c5e4-4d83-9cf5-9addf22f86b7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:30.570396 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a179ab6-c5e4-4d83-9cf5-9addf22f86b7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:30.570835 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a179ab6-c5e4-4d83-9cf5-9addf22f86b7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:30.571527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a179ab6-c5e4-4d83-9cf5-9addf22f86b7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:30.571977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a179ab6-c5e4-4d83-9cf5-9addf22f86b7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:30.572402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a179ab6-c5e4-4d83-9cf5-9addf22f86b7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:30.708558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5a179ab6-c5e4-4d83-9cf5-9addf22f86b7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:30.716272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5a179ab6-c5e4-4d83-9cf5-9addf22f86b7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.156961 Sep 08 13:59:30.716836 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 458/799] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:30 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:30.918635 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6a1edc4f-779b-42c3-83e2-678b670ef6a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:30.919282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6a1edc4f-779b-42c3-83e2-678b670ef6a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:30.925442 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a1edc4f-779b-42c3-83e2-678b670ef6a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:30.925929 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a1edc4f-779b-42c3-83e2-678b670ef6a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:30.926394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a1edc4f-779b-42c3-83e2-678b670ef6a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:30.927087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a1edc4f-779b-42c3-83e2-678b670ef6a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:30.927542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a1edc4f-779b-42c3-83e2-678b670ef6a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:30.927966 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a1edc4f-779b-42c3-83e2-678b670ef6a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:31.058755 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6a1edc4f-779b-42c3-83e2-678b670ef6a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:31.067254 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6a1edc4f-779b-42c3-83e2-678b670ef6a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.152514 Sep 08 13:59:31.067825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 342/800] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:30 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:31.321636 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b9511994-1cfd-4197-92b0-82f9d267db55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:31.322356 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b9511994-1cfd-4197-92b0-82f9d267db55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:31.328900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b9511994-1cfd-4197-92b0-82f9d267db55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:31.329387 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b9511994-1cfd-4197-92b0-82f9d267db55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:31.329822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b9511994-1cfd-4197-92b0-82f9d267db55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:31.330547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b9511994-1cfd-4197-92b0-82f9d267db55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:31.331032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b9511994-1cfd-4197-92b0-82f9d267db55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:31.331454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b9511994-1cfd-4197-92b0-82f9d267db55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:31.459452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b9511994-1cfd-4197-92b0-82f9d267db55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:31.468679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b9511994-1cfd-4197-92b0-82f9d267db55 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.151017 Sep 08 13:59:31.469278 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 459/801] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:31 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:31.551401 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0b00606a-dbcd-4acd-b3d6-892e627475c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:31.553537 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0b00606a-dbcd-4acd-b3d6-892e627475c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:31.560114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0b00606a-dbcd-4acd-b3d6-892e627475c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:31.564909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0b00606a-dbcd-4acd-b3d6-892e627475c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:31.564909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0b00606a-dbcd-4acd-b3d6-892e627475c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:31.564909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0b00606a-dbcd-4acd-b3d6-892e627475c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:31.564909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0b00606a-dbcd-4acd-b3d6-892e627475c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:31.565285 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0b00606a-dbcd-4acd-b3d6-892e627475c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:31.593453 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0b00606a-dbcd-4acd-b3d6-892e627475c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046009 Sep 08 13:59:31.594018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 343/802] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:31 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:31.739764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-36c33830-c489-495a-9eb9-ddaa7aec1140 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:31.740325 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-36c33830-c489-495a-9eb9-ddaa7aec1140 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:31.746324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-36c33830-c489-495a-9eb9-ddaa7aec1140 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:31.747206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-36c33830-c489-495a-9eb9-ddaa7aec1140 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:31.748458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-36c33830-c489-495a-9eb9-ddaa7aec1140 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:31.748580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-36c33830-c489-495a-9eb9-ddaa7aec1140 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:31.748979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-36c33830-c489-495a-9eb9-ddaa7aec1140 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:31.749407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-36c33830-c489-495a-9eb9-ddaa7aec1140 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:31.890539 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-36c33830-c489-495a-9eb9-ddaa7aec1140 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:31.898519 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-36c33830-c489-495a-9eb9-ddaa7aec1140 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.162755 Sep 08 13:59:31.902463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 460/803] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:31 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:32.090514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5710bc32-5b62-4e90-9c34-3d06ef993805 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:32.091119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5710bc32-5b62-4e90-9c34-3d06ef993805 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:32.099100 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5710bc32-5b62-4e90-9c34-3d06ef993805 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:32.099633 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5710bc32-5b62-4e90-9c34-3d06ef993805 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:32.100067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5710bc32-5b62-4e90-9c34-3d06ef993805 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:32.100765 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5710bc32-5b62-4e90-9c34-3d06ef993805 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:32.101222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5710bc32-5b62-4e90-9c34-3d06ef993805 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:32.101646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5710bc32-5b62-4e90-9c34-3d06ef993805 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:32.236107 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5710bc32-5b62-4e90-9c34-3d06ef993805 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:32.244115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5710bc32-5b62-4e90-9c34-3d06ef993805 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.157389 Sep 08 13:59:32.244729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 344/804] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:32 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:32.492606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7f458ac6-fb95-4e0e-8fff-6acb2648ff26 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:32.493210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7f458ac6-fb95-4e0e-8fff-6acb2648ff26 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:32.500651 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f458ac6-fb95-4e0e-8fff-6acb2648ff26 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:32.501356 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f458ac6-fb95-4e0e-8fff-6acb2648ff26 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:32.501741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f458ac6-fb95-4e0e-8fff-6acb2648ff26 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:32.502488 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f458ac6-fb95-4e0e-8fff-6acb2648ff26 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:32.503044 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f458ac6-fb95-4e0e-8fff-6acb2648ff26 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:32.503487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f458ac6-fb95-4e0e-8fff-6acb2648ff26 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:32.605928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a52605a4-e0d8-43d7-91bf-f05e78ebfe38 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:32.608616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a52605a4-e0d8-43d7-91bf-f05e78ebfe38 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:32.615295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a52605a4-e0d8-43d7-91bf-f05e78ebfe38 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:32.615766 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a52605a4-e0d8-43d7-91bf-f05e78ebfe38 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:32.616213 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a52605a4-e0d8-43d7-91bf-f05e78ebfe38 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:32.616877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a52605a4-e0d8-43d7-91bf-f05e78ebfe38 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:32.617323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a52605a4-e0d8-43d7-91bf-f05e78ebfe38 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:32.617742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a52605a4-e0d8-43d7-91bf-f05e78ebfe38 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:32.639714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7f458ac6-fb95-4e0e-8fff-6acb2648ff26 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:32.649555 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a52605a4-e0d8-43d7-91bf-f05e78ebfe38 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047621 Sep 08 13:59:32.650349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 345/805] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:32 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:59:32.651631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7f458ac6-fb95-4e0e-8fff-6acb2648ff26 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.162509 Sep 08 13:59:32.652245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 461/806] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:32 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:32.922849 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5b247999-e757-414b-a6b5-25b5de46c94f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:32.923601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5b247999-e757-414b-a6b5-25b5de46c94f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:32.929541 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5b247999-e757-414b-a6b5-25b5de46c94f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:32.930027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5b247999-e757-414b-a6b5-25b5de46c94f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:32.930495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5b247999-e757-414b-a6b5-25b5de46c94f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:32.931191 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5b247999-e757-414b-a6b5-25b5de46c94f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:32.931642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5b247999-e757-414b-a6b5-25b5de46c94f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:32.932068 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5b247999-e757-414b-a6b5-25b5de46c94f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:33.066703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5b247999-e757-414b-a6b5-25b5de46c94f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:33.074346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5b247999-e757-414b-a6b5-25b5de46c94f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.156049 Sep 08 13:59:33.074914 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 346/807] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:32 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:33.268815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-06e2effd-ca73-4f83-a956-b0555ba41afc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:33.269530 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-06e2effd-ca73-4f83-a956-b0555ba41afc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:33.275696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06e2effd-ca73-4f83-a956-b0555ba41afc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:33.276185 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06e2effd-ca73-4f83-a956-b0555ba41afc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:33.276622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06e2effd-ca73-4f83-a956-b0555ba41afc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:33.277319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06e2effd-ca73-4f83-a956-b0555ba41afc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:33.277777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06e2effd-ca73-4f83-a956-b0555ba41afc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:33.278344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06e2effd-ca73-4f83-a956-b0555ba41afc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:33.426529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-06e2effd-ca73-4f83-a956-b0555ba41afc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:33.434444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-06e2effd-ca73-4f83-a956-b0555ba41afc tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.169995 Sep 08 13:59:33.435039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 462/808] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:33 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:33.662237 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f0063346-400a-4c4c-8b86-b10d0e80b2ee tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:33.664493 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f0063346-400a-4c4c-8b86-b10d0e80b2ee tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:33.671274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f0063346-400a-4c4c-8b86-b10d0e80b2ee tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:33.671746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f0063346-400a-4c4c-8b86-b10d0e80b2ee tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:33.672208 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f0063346-400a-4c4c-8b86-b10d0e80b2ee tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:33.672903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f0063346-400a-4c4c-8b86-b10d0e80b2ee tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:33.673356 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f0063346-400a-4c4c-8b86-b10d0e80b2ee tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:33.673869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f0063346-400a-4c4c-8b86-b10d0e80b2ee tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:33.678850 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0321588f-eb51-4b2b-8d16-5acfc088d604 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:33.679554 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0321588f-eb51-4b2b-8d16-5acfc088d604 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:33.685976 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0321588f-eb51-4b2b-8d16-5acfc088d604 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:33.686539 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0321588f-eb51-4b2b-8d16-5acfc088d604 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:33.687512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0321588f-eb51-4b2b-8d16-5acfc088d604 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:33.688223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0321588f-eb51-4b2b-8d16-5acfc088d604 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:33.688676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0321588f-eb51-4b2b-8d16-5acfc088d604 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:33.689104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0321588f-eb51-4b2b-8d16-5acfc088d604 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:33.691513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:33.705264 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f0063346-400a-4c4c-8b86-b10d0e80b2ee tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047080 Sep 08 13:59:33.705841 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 347/809] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:33 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:33.826773 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0321588f-eb51-4b2b-8d16-5acfc088d604 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:33.835671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0321588f-eb51-4b2b-8d16-5acfc088d604 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.160681 Sep 08 13:59:33.836236 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 463/810] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:33 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:59:34.101228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a66ae096-0542-4ecc-b962-61ddb19d917b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:34.101852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a66ae096-0542-4ecc-b962-61ddb19d917b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:34.108415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a66ae096-0542-4ecc-b962-61ddb19d917b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:34.108920 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a66ae096-0542-4ecc-b962-61ddb19d917b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:34.109341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a66ae096-0542-4ecc-b962-61ddb19d917b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:34.110009 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a66ae096-0542-4ecc-b962-61ddb19d917b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:34.110477 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a66ae096-0542-4ecc-b962-61ddb19d917b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:34.110889 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a66ae096-0542-4ecc-b962-61ddb19d917b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:34.245521 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a66ae096-0542-4ecc-b962-61ddb19d917b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:34.253530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a66ae096-0542-4ecc-b962-61ddb19d917b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.156217 Sep 08 13:59:34.254105 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 348/811] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:34 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:34.459099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7f1b2e4f-c760-4f5f-9228-78eefd2791c7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:34.459744 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7f1b2e4f-c760-4f5f-9228-78eefd2791c7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:34.465915 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f1b2e4f-c760-4f5f-9228-78eefd2791c7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:34.466428 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f1b2e4f-c760-4f5f-9228-78eefd2791c7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:34.466864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f1b2e4f-c760-4f5f-9228-78eefd2791c7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:34.467578 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f1b2e4f-c760-4f5f-9228-78eefd2791c7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:34.468065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f1b2e4f-c760-4f5f-9228-78eefd2791c7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:34.468516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f1b2e4f-c760-4f5f-9228-78eefd2791c7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:34.601289 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7f1b2e4f-c760-4f5f-9228-78eefd2791c7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:34.609138 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7f1b2e4f-c760-4f5f-9228-78eefd2791c7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.153957 Sep 08 13:59:34.609638 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 464/812] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:34 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:34.718574 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-64abd8b4-119b-47eb-bf5c-5036006cdda6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:34.720710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-64abd8b4-119b-47eb-bf5c-5036006cdda6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:34.726899 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64abd8b4-119b-47eb-bf5c-5036006cdda6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:34.727383 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64abd8b4-119b-47eb-bf5c-5036006cdda6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:34.727818 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64abd8b4-119b-47eb-bf5c-5036006cdda6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:34.728510 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64abd8b4-119b-47eb-bf5c-5036006cdda6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:34.728964 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64abd8b4-119b-47eb-bf5c-5036006cdda6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:34.729387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64abd8b4-119b-47eb-bf5c-5036006cdda6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:34.760457 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-64abd8b4-119b-47eb-bf5c-5036006cdda6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046252 Sep 08 13:59:34.761023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 349/813] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:34 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:34.860111 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-00e517b4-c06b-494c-93ec-a2040504eb94 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:34.860623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-00e517b4-c06b-494c-93ec-a2040504eb94 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:34.868077 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-00e517b4-c06b-494c-93ec-a2040504eb94 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:34.868669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-00e517b4-c06b-494c-93ec-a2040504eb94 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:34.869171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-00e517b4-c06b-494c-93ec-a2040504eb94 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:34.870005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-00e517b4-c06b-494c-93ec-a2040504eb94 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:34.870593 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-00e517b4-c06b-494c-93ec-a2040504eb94 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:34.871064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-00e517b4-c06b-494c-93ec-a2040504eb94 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:35.002383 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-00e517b4-c06b-494c-93ec-a2040504eb94 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:35.010467 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-00e517b4-c06b-494c-93ec-a2040504eb94 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.155025 Sep 08 13:59:35.011021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 465/814] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:34 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:35.278253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-71eb1756-67ef-4a3d-92d0-ff6a580351d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:35.278865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-71eb1756-67ef-4a3d-92d0-ff6a580351d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:35.285503 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71eb1756-67ef-4a3d-92d0-ff6a580351d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:35.285978 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71eb1756-67ef-4a3d-92d0-ff6a580351d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:35.286441 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71eb1756-67ef-4a3d-92d0-ff6a580351d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:35.287117 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71eb1756-67ef-4a3d-92d0-ff6a580351d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:35.287560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71eb1756-67ef-4a3d-92d0-ff6a580351d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:35.287976 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71eb1756-67ef-4a3d-92d0-ff6a580351d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:35.422096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-71eb1756-67ef-4a3d-92d0-ff6a580351d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:35.429711 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-71eb1756-67ef-4a3d-92d0-ff6a580351d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.155617 Sep 08 13:59:35.430301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 350/815] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:35 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:35.633117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f4e72875-5c48-475b-9ec0-dd68440a7912 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:35.633780 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f4e72875-5c48-475b-9ec0-dd68440a7912 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:35.640728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f4e72875-5c48-475b-9ec0-dd68440a7912 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:35.641022 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f4e72875-5c48-475b-9ec0-dd68440a7912 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:35.641468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f4e72875-5c48-475b-9ec0-dd68440a7912 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:35.642190 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f4e72875-5c48-475b-9ec0-dd68440a7912 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:35.642677 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f4e72875-5c48-475b-9ec0-dd68440a7912 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:35.643105 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f4e72875-5c48-475b-9ec0-dd68440a7912 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:35.773394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3f13795c-569b-444d-988c-b678e6783b9f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:35.773880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f4e72875-5c48-475b-9ec0-dd68440a7912 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:35.775632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3f13795c-569b-444d-988c-b678e6783b9f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:35.783204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f4e72875-5c48-475b-9ec0-dd68440a7912 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.153960 Sep 08 13:59:35.784765 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3f13795c-569b-444d-988c-b678e6783b9f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:35.785251 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3f13795c-569b-444d-988c-b678e6783b9f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:35.785676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3f13795c-569b-444d-988c-b678e6783b9f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:35.785894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 466/816] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:35 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:35.786835 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3f13795c-569b-444d-988c-b678e6783b9f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:35.787352 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3f13795c-569b-444d-988c-b678e6783b9f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:35.787852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3f13795c-569b-444d-988c-b678e6783b9f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:35.818277 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3f13795c-569b-444d-988c-b678e6783b9f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049089 Sep 08 13:59:35.818835 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 351/817] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:35 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:36.034698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5cfadb0c-3c6e-4458-aafd-6b2a20e32b01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:36.035238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5cfadb0c-3c6e-4458-aafd-6b2a20e32b01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:36.041984 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5cfadb0c-3c6e-4458-aafd-6b2a20e32b01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:36.042507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5cfadb0c-3c6e-4458-aafd-6b2a20e32b01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:36.042949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5cfadb0c-3c6e-4458-aafd-6b2a20e32b01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:36.043810 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5cfadb0c-3c6e-4458-aafd-6b2a20e32b01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:36.044270 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5cfadb0c-3c6e-4458-aafd-6b2a20e32b01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:36.044697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5cfadb0c-3c6e-4458-aafd-6b2a20e32b01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:36.182641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5cfadb0c-3c6e-4458-aafd-6b2a20e32b01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:36.191335 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5cfadb0c-3c6e-4458-aafd-6b2a20e32b01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.160589 Sep 08 13:59:36.191902 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 467/818] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:36 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:36.454608 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-223a3ffe-3660-47d5-8d97-661272294adb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:36.455269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-223a3ffe-3660-47d5-8d97-661272294adb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:36.461596 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-223a3ffe-3660-47d5-8d97-661272294adb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:36.462081 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-223a3ffe-3660-47d5-8d97-661272294adb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:36.462548 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-223a3ffe-3660-47d5-8d97-661272294adb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:36.463239 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-223a3ffe-3660-47d5-8d97-661272294adb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:36.463688 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-223a3ffe-3660-47d5-8d97-661272294adb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:36.464108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-223a3ffe-3660-47d5-8d97-661272294adb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:36.601528 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-223a3ffe-3660-47d5-8d97-661272294adb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:36.610462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-223a3ffe-3660-47d5-8d97-661272294adb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.160885 Sep 08 13:59:36.611250 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 352/819] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:36 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:36.807013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-30c22cfe-2df8-4938-87b7-91b2a55fba2e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:36.807546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-30c22cfe-2df8-4938-87b7-91b2a55fba2e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:36.813574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-30c22cfe-2df8-4938-87b7-91b2a55fba2e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:36.814052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-30c22cfe-2df8-4938-87b7-91b2a55fba2e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:36.814529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-30c22cfe-2df8-4938-87b7-91b2a55fba2e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:36.815218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-30c22cfe-2df8-4938-87b7-91b2a55fba2e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:36.815671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-30c22cfe-2df8-4938-87b7-91b2a55fba2e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:36.816095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-30c22cfe-2df8-4938-87b7-91b2a55fba2e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:36.831667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ef8e2aa5-6fd9-4f86-a20f-d8f0e032a8b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:36.833882 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ef8e2aa5-6fd9-4f86-a20f-d8f0e032a8b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:36.840453 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef8e2aa5-6fd9-4f86-a20f-d8f0e032a8b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:36.840963 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef8e2aa5-6fd9-4f86-a20f-d8f0e032a8b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:36.841428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef8e2aa5-6fd9-4f86-a20f-d8f0e032a8b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:36.842118 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef8e2aa5-6fd9-4f86-a20f-d8f0e032a8b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:36.843184 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef8e2aa5-6fd9-4f86-a20f-d8f0e032a8b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:36.843574 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef8e2aa5-6fd9-4f86-a20f-d8f0e032a8b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:36.874833 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ef8e2aa5-6fd9-4f86-a20f-d8f0e032a8b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047117 Sep 08 13:59:36.875401 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 353/820] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:36 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:59:36.950680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-30c22cfe-2df8-4938-87b7-91b2a55fba2e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:36.958743 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-30c22cfe-2df8-4938-87b7-91b2a55fba2e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.155750 Sep 08 13:59:36.959314 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 468/821] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:36 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:37.215866 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-24255867-c6fb-4b71-990c-027b77a70a9a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:37.216488 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-24255867-c6fb-4b71-990c-027b77a70a9a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:37.223306 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-24255867-c6fb-4b71-990c-027b77a70a9a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:37.223792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-24255867-c6fb-4b71-990c-027b77a70a9a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:37.224234 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-24255867-c6fb-4b71-990c-027b77a70a9a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:37.224977 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-24255867-c6fb-4b71-990c-027b77a70a9a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:37.225429 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-24255867-c6fb-4b71-990c-027b77a70a9a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:37.225859 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-24255867-c6fb-4b71-990c-027b77a70a9a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:37.360852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-24255867-c6fb-4b71-990c-027b77a70a9a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:37.370054 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-24255867-c6fb-4b71-990c-027b77a70a9a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.157755 Sep 08 13:59:37.378309 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 354/822] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:37 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:37.632915 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-42edf6fd-c63b-4e91-bd82-8b9c3f408a0b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:37.633459 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-42edf6fd-c63b-4e91-bd82-8b9c3f408a0b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:37.640818 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-42edf6fd-c63b-4e91-bd82-8b9c3f408a0b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:37.641308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-42edf6fd-c63b-4e91-bd82-8b9c3f408a0b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:37.641749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-42edf6fd-c63b-4e91-bd82-8b9c3f408a0b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:37.642481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-42edf6fd-c63b-4e91-bd82-8b9c3f408a0b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:37.642939 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-42edf6fd-c63b-4e91-bd82-8b9c3f408a0b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:37.643367 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-42edf6fd-c63b-4e91-bd82-8b9c3f408a0b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:37.778799 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-42edf6fd-c63b-4e91-bd82-8b9c3f408a0b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:37.786787 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-42edf6fd-c63b-4e91-bd82-8b9c3f408a0b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.157830 Sep 08 13:59:37.787318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 469/823] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:37 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:37.888310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fb28af18-2075-42be-b5a4-2d124bd4f748 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:37.890508 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fb28af18-2075-42be-b5a4-2d124bd4f748 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:37.897323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb28af18-2075-42be-b5a4-2d124bd4f748 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:37.897803 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb28af18-2075-42be-b5a4-2d124bd4f748 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:37.898281 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb28af18-2075-42be-b5a4-2d124bd4f748 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:37.899225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb28af18-2075-42be-b5a4-2d124bd4f748 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:37.899680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb28af18-2075-42be-b5a4-2d124bd4f748 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:37.900104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb28af18-2075-42be-b5a4-2d124bd4f748 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:37.930665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fb28af18-2075-42be-b5a4-2d124bd4f748 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046393 Sep 08 13:59:37.931225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 355/824] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:37 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:37.983644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-293d1a7a-187a-4ca9-9683-19a69ffed610 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:37.984261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-293d1a7a-187a-4ca9-9683-19a69ffed610 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:37.990557 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-293d1a7a-187a-4ca9-9683-19a69ffed610 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:37.991050 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-293d1a7a-187a-4ca9-9683-19a69ffed610 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:37.991492 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-293d1a7a-187a-4ca9-9683-19a69ffed610 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:37.992187 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-293d1a7a-187a-4ca9-9683-19a69ffed610 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:37.992725 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-293d1a7a-187a-4ca9-9683-19a69ffed610 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:37.993076 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-293d1a7a-187a-4ca9-9683-19a69ffed610 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:38.123347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-293d1a7a-187a-4ca9-9683-19a69ffed610 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:38.131323 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-293d1a7a-187a-4ca9-9683-19a69ffed610 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.151863 Sep 08 13:59:38.131620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 470/825] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:37 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:38.401088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-24b4d125-50d1-4eb8-9ce6-67974335a63a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:38.401780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-24b4d125-50d1-4eb8-9ce6-67974335a63a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:38.408714 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-24b4d125-50d1-4eb8-9ce6-67974335a63a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:38.409193 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-24b4d125-50d1-4eb8-9ce6-67974335a63a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:38.409685 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-24b4d125-50d1-4eb8-9ce6-67974335a63a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:38.410457 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-24b4d125-50d1-4eb8-9ce6-67974335a63a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:38.410918 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-24b4d125-50d1-4eb8-9ce6-67974335a63a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:38.411342 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-24b4d125-50d1-4eb8-9ce6-67974335a63a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:38.548598 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-24b4d125-50d1-4eb8-9ce6-67974335a63a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:38.557918 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-24b4d125-50d1-4eb8-9ce6-67974335a63a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.159667 Sep 08 13:59:38.557918 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 356/826] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:38 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:38.809322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-319a1345-2f15-4cd9-838f-c961e08180eb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:38.809957 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-319a1345-2f15-4cd9-838f-c961e08180eb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:38.817195 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-319a1345-2f15-4cd9-838f-c961e08180eb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:38.817846 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-319a1345-2f15-4cd9-838f-c961e08180eb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:38.818438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-319a1345-2f15-4cd9-838f-c961e08180eb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:38.819146 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-319a1345-2f15-4cd9-838f-c961e08180eb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:38.819690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-319a1345-2f15-4cd9-838f-c961e08180eb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:38.820139 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-319a1345-2f15-4cd9-838f-c961e08180eb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:38.943356 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cb8793f5-e261-4e67-85cb-44998251df4f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:38.945979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cb8793f5-e261-4e67-85cb-44998251df4f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:38.956624 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb8793f5-e261-4e67-85cb-44998251df4f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:38.957114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb8793f5-e261-4e67-85cb-44998251df4f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:38.957757 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb8793f5-e261-4e67-85cb-44998251df4f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:38.958670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb8793f5-e261-4e67-85cb-44998251df4f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:38.959124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb8793f5-e261-4e67-85cb-44998251df4f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:38.959549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb8793f5-e261-4e67-85cb-44998251df4f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:38.970172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-319a1345-2f15-4cd9-838f-c961e08180eb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:38.979172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-319a1345-2f15-4cd9-838f-c961e08180eb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.173936 Sep 08 13:59:38.979746 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 471/827] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:38 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:39.000251 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cb8793f5-e261-4e67-85cb-44998251df4f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061036 Sep 08 13:59:39.000839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 357/828] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:38 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:59:39.155523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4026da3b-9ee9-461e-b33b-b80e3ba19a90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:39.156485 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4026da3b-9ee9-461e-b33b-b80e3ba19a90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:39.162624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4026da3b-9ee9-461e-b33b-b80e3ba19a90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:39.163139 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4026da3b-9ee9-461e-b33b-b80e3ba19a90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:39.163583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4026da3b-9ee9-461e-b33b-b80e3ba19a90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:39.164289 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4026da3b-9ee9-461e-b33b-b80e3ba19a90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:39.164752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4026da3b-9ee9-461e-b33b-b80e3ba19a90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:39.165178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4026da3b-9ee9-461e-b33b-b80e3ba19a90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:39.309873 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4026da3b-9ee9-461e-b33b-b80e3ba19a90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:39.317693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4026da3b-9ee9-461e-b33b-b80e3ba19a90 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1680 microversion: 2.1 time: 0.166537 Sep 08 13:59:39.318406 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 472/829] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:39 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1680 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:39.580520 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d14939ae-50e1-4edd-860c-fff18136bc88 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:39.581185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d14939ae-50e1-4edd-860c-fff18136bc88 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:39.590820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d14939ae-50e1-4edd-860c-fff18136bc88 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:39.591330 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d14939ae-50e1-4edd-860c-fff18136bc88 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:39.592307 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d14939ae-50e1-4edd-860c-fff18136bc88 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:39.593091 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d14939ae-50e1-4edd-860c-fff18136bc88 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:39.593569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d14939ae-50e1-4edd-860c-fff18136bc88 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:39.594019 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d14939ae-50e1-4edd-860c-fff18136bc88 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:39.799467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d14939ae-50e1-4edd-860c-fff18136bc88 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:39.813366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d14939ae-50e1-4edd-860c-fff18136bc88 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.236874 Sep 08 13:59:39.814204 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 358/830] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:39 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:39.896864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-21b678fe-76ac-4d99-a8e7-8d543a1a1854 req-84a69ece-2b89-48aa-b4ca-61b33313063f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "483c7eb7-3827-403f-84f3-c7c65dcbe574", "name": "network-vif-deleted", "tag": "207fb2dc-4cc9-4491-b6db-03ae99699ac3"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:59:39.907823 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-21b678fe-76ac-4d99-a8e7-8d543a1a1854 req-84a69ece-2b89-48aa-b4ca-61b33313063f service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:39.908301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-21b678fe-76ac-4d99-a8e7-8d543a1a1854 req-84a69ece-2b89-48aa-b4ca-61b33313063f service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:39.909094 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-21b678fe-76ac-4d99-a8e7-8d543a1a1854 req-84a69ece-2b89-48aa-b4ca-61b33313063f service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:39.922236 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-21b678fe-76ac-4d99-a8e7-8d543a1a1854 req-84a69ece-2b89-48aa-b4ca-61b33313063f service nova] Creating event network-vif-deleted:207fb2dc-4cc9-4491-b6db-03ae99699ac3 for instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 on hv-cinder-85646 Sep 08 13:59:39.927565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-21b678fe-76ac-4d99-a8e7-8d543a1a1854 req-84a69ece-2b89-48aa-b4ca-61b33313063f service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.106848 Sep 08 13:59:39.928181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 473/831] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:59:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:59:40.002611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4c3a54a0-f9c3-4ac1-8805-954fd9a4dca2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:40.003926 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4c3a54a0-f9c3-4ac1-8805-954fd9a4dca2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:40.014442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-de9294c8-73c2-4460-b137-3752cd051247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:40.016498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-de9294c8-73c2-4460-b137-3752cd051247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:40.021185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4c3a54a0-f9c3-4ac1-8805-954fd9a4dca2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:40.021681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4c3a54a0-f9c3-4ac1-8805-954fd9a4dca2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:40.022117 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4c3a54a0-f9c3-4ac1-8805-954fd9a4dca2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:40.022847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4c3a54a0-f9c3-4ac1-8805-954fd9a4dca2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:40.023301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4c3a54a0-f9c3-4ac1-8805-954fd9a4dca2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:40.023797 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4c3a54a0-f9c3-4ac1-8805-954fd9a4dca2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:40.033916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de9294c8-73c2-4460-b137-3752cd051247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:40.034449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de9294c8-73c2-4460-b137-3752cd051247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:40.034893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de9294c8-73c2-4460-b137-3752cd051247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:40.035606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de9294c8-73c2-4460-b137-3752cd051247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:40.036067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de9294c8-73c2-4460-b137-3752cd051247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:40.036497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de9294c8-73c2-4460-b137-3752cd051247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:40.039296 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:40.082118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-de9294c8-73c2-4460-b137-3752cd051247 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.072664 Sep 08 13:59:40.083016 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 474/832] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:40 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:59:40.176908 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4c3a54a0-f9c3-4ac1-8805-954fd9a4dca2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:40.184986 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4c3a54a0-f9c3-4ac1-8805-954fd9a4dca2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.186156 Sep 08 13:59:40.185591 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 359/833] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:39 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:40.342642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4e32ea76-745f-43b8-9e33-5f068b106390 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:40.343446 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4e32ea76-745f-43b8-9e33-5f068b106390 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:40.350357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4e32ea76-745f-43b8-9e33-5f068b106390 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:40.350856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4e32ea76-745f-43b8-9e33-5f068b106390 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:40.351299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4e32ea76-745f-43b8-9e33-5f068b106390 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:40.352009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4e32ea76-745f-43b8-9e33-5f068b106390 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:40.352473 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4e32ea76-745f-43b8-9e33-5f068b106390 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:40.352953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4e32ea76-745f-43b8-9e33-5f068b106390 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:40.421262 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4e32ea76-745f-43b8-9e33-5f068b106390 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:40.429760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4e32ea76-745f-43b8-9e33-5f068b106390 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1613 microversion: 2.1 time: 0.091302 Sep 08 13:59:40.430379 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 475/834] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:40 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1613 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:40.841845 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1b9380aa-1031-453a-8f98-bc5ef95654b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:40.843093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1b9380aa-1031-453a-8f98-bc5ef95654b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:40.848812 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b9380aa-1031-453a-8f98-bc5ef95654b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:40.849295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b9380aa-1031-453a-8f98-bc5ef95654b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:40.849737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b9380aa-1031-453a-8f98-bc5ef95654b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:40.850472 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b9380aa-1031-453a-8f98-bc5ef95654b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:40.850939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b9380aa-1031-453a-8f98-bc5ef95654b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:40.851363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b9380aa-1031-453a-8f98-bc5ef95654b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:40.990795 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1b9380aa-1031-453a-8f98-bc5ef95654b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:41.008223 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1b9380aa-1031-453a-8f98-bc5ef95654b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.172529 Sep 08 13:59:41.008889 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 360/835] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:40 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:41.095197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-472e3ae0-0487-462a-8729-48e1a949e838 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:41.098070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-472e3ae0-0487-462a-8729-48e1a949e838 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:41.104737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-472e3ae0-0487-462a-8729-48e1a949e838 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:41.105443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-472e3ae0-0487-462a-8729-48e1a949e838 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:41.105894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-472e3ae0-0487-462a-8729-48e1a949e838 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:41.107173 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-472e3ae0-0487-462a-8729-48e1a949e838 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:41.107640 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-472e3ae0-0487-462a-8729-48e1a949e838 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:41.108074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-472e3ae0-0487-462a-8729-48e1a949e838 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:41.143353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-472e3ae0-0487-462a-8729-48e1a949e838 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052428 Sep 08 13:59:41.148057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 476/836] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:41 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:41.216739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-75278bdc-2004-4fff-8d2c-6dacf7b80162 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:41.217355 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-75278bdc-2004-4fff-8d2c-6dacf7b80162 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:41.228767 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-75278bdc-2004-4fff-8d2c-6dacf7b80162 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:41.231215 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-75278bdc-2004-4fff-8d2c-6dacf7b80162 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:41.233278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-75278bdc-2004-4fff-8d2c-6dacf7b80162 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:41.235221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-75278bdc-2004-4fff-8d2c-6dacf7b80162 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:41.236883 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-75278bdc-2004-4fff-8d2c-6dacf7b80162 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:41.237521 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-75278bdc-2004-4fff-8d2c-6dacf7b80162 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:41.440656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-75278bdc-2004-4fff-8d2c-6dacf7b80162 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:41.451713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-75278bdc-2004-4fff-8d2c-6dacf7b80162 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.247639 Sep 08 13:59:41.452918 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 361/837] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:41 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:41.453726 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-26d5b265-04fa-40f7-b523-17fbdfeb349f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:41.454381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-26d5b265-04fa-40f7-b523-17fbdfeb349f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:41.460526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26d5b265-04fa-40f7-b523-17fbdfeb349f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:41.461033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26d5b265-04fa-40f7-b523-17fbdfeb349f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:41.461474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26d5b265-04fa-40f7-b523-17fbdfeb349f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:41.462199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26d5b265-04fa-40f7-b523-17fbdfeb349f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:41.462862 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26d5b265-04fa-40f7-b523-17fbdfeb349f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:41.463328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26d5b265-04fa-40f7-b523-17fbdfeb349f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:41.496447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26d5b265-04fa-40f7-b523-17fbdfeb349f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:41.496940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26d5b265-04fa-40f7-b523-17fbdfeb349f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:41.497993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26d5b265-04fa-40f7-b523-17fbdfeb349f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:41.498539 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-26d5b265-04fa-40f7-b523-17fbdfeb349f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lazy-loading 'fault' on Instance uuid 483c7eb7-3827-403f-84f3-c7c65dcbe574 {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 13:59:41.540548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-26d5b265-04fa-40f7-b523-17fbdfeb349f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:41.549101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-26d5b265-04fa-40f7-b523-17fbdfeb349f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1331 microversion: 2.1 time: 0.099216 Sep 08 13:59:41.549731 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 477/838] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:41 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1331 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:59:42.002311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-debbd53b-e5ad-4476-b31d-e0d4dd69aafe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:42.002922 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-debbd53b-e5ad-4476-b31d-e0d4dd69aafe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:42.010815 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-debbd53b-e5ad-4476-b31d-e0d4dd69aafe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:42.011451 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-debbd53b-e5ad-4476-b31d-e0d4dd69aafe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:42.011747 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-debbd53b-e5ad-4476-b31d-e0d4dd69aafe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:42.012698 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-debbd53b-e5ad-4476-b31d-e0d4dd69aafe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:42.014367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-debbd53b-e5ad-4476-b31d-e0d4dd69aafe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:42.014728 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-debbd53b-e5ad-4476-b31d-e0d4dd69aafe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:42.033819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ad8af6ff-f643-4ee4-b098-5e2361fb3014 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:42.034459 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ad8af6ff-f643-4ee4-b098-5e2361fb3014 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:42.040541 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad8af6ff-f643-4ee4-b098-5e2361fb3014 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:42.041033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad8af6ff-f643-4ee4-b098-5e2361fb3014 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:42.041473 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad8af6ff-f643-4ee4-b098-5e2361fb3014 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:42.042283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad8af6ff-f643-4ee4-b098-5e2361fb3014 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:42.042748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad8af6ff-f643-4ee4-b098-5e2361fb3014 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:42.043185 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad8af6ff-f643-4ee4-b098-5e2361fb3014 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:42.163649 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-debbd53b-e5ad-4476-b31d-e0d4dd69aafe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:42.172687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-debbd53b-e5ad-4476-b31d-e0d4dd69aafe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1592 microversion: 2.1 time: 0.173855 Sep 08 13:59:42.173457 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 362/839] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:41 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1592 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:42.179661 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-50658c13-f89f-4670-a779-43fe7337060f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:42.182379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-50658c13-f89f-4670-a779-43fe7337060f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:42.191369 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-50658c13-f89f-4670-a779-43fe7337060f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:42.192165 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-50658c13-f89f-4670-a779-43fe7337060f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:42.192861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-50658c13-f89f-4670-a779-43fe7337060f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:42.193400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ad8af6ff-f643-4ee4-b098-5e2361fb3014 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:42.194161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-50658c13-f89f-4670-a779-43fe7337060f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:42.195043 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-50658c13-f89f-4670-a779-43fe7337060f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:42.195761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-50658c13-f89f-4670-a779-43fe7337060f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:42.209218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ad8af6ff-f643-4ee4-b098-5e2361fb3014 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.179264 Sep 08 13:59:42.210664 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 478/840] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:42 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:59:42.247075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-50658c13-f89f-4670-a779-43fe7337060f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.072043 Sep 08 13:59:42.248546 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 363/841] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:42 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:42.479577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-af1d1f89-54d5-4766-ad25-d886a4c9079b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:42.480188 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-af1d1f89-54d5-4766-ad25-d886a4c9079b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:42.486422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af1d1f89-54d5-4766-ad25-d886a4c9079b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:42.486911 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af1d1f89-54d5-4766-ad25-d886a4c9079b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:42.487370 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af1d1f89-54d5-4766-ad25-d886a4c9079b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:42.488071 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af1d1f89-54d5-4766-ad25-d886a4c9079b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:42.488518 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af1d1f89-54d5-4766-ad25-d886a4c9079b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:42.488932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af1d1f89-54d5-4766-ad25-d886a4c9079b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:42.606538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-0d9b3b47-e8c0-4f96-af33-9822d949655a req-1fe18315-2330-43b1-99e0-7f702b40e5c1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d1062abf-7525-48c5-a9cb-91da3855fd1b", "name": "network-vif-plugged", "status": "completed", "tag": "2d04f3d2-99c9-4712-bc13-6ebdffcb949f"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:59:42.617036 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-0d9b3b47-e8c0-4f96-af33-9822d949655a req-1fe18315-2330-43b1-99e0-7f702b40e5c1 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:42.617578 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-0d9b3b47-e8c0-4f96-af33-9822d949655a req-1fe18315-2330-43b1-99e0-7f702b40e5c1 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:42.618003 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-0d9b3b47-e8c0-4f96-af33-9822d949655a req-1fe18315-2330-43b1-99e0-7f702b40e5c1 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:42.623487 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:42.637120 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-0d9b3b47-e8c0-4f96-af33-9822d949655a req-1fe18315-2330-43b1-99e0-7f702b40e5c1 service nova] Creating event network-vif-plugged:2d04f3d2-99c9-4712-bc13-6ebdffcb949f for instance d1062abf-7525-48c5-a9cb-91da3855fd1b on hv-cinder-85646 Sep 08 13:59:42.649140 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-0d9b3b47-e8c0-4f96-af33-9822d949655a req-1fe18315-2330-43b1-99e0-7f702b40e5c1 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.111485 Sep 08 13:59:42.651180 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 364/842] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:59:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:59:42.664473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4d2e2cfb-1446-4a54-850e-dcc7698890d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:42.664623 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4d2e2cfb-1446-4a54-850e-dcc7698890d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:42.676743 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d2e2cfb-1446-4a54-850e-dcc7698890d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:42.677666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d2e2cfb-1446-4a54-850e-dcc7698890d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:42.678468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d2e2cfb-1446-4a54-850e-dcc7698890d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:42.679750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d2e2cfb-1446-4a54-850e-dcc7698890d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:42.680557 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d2e2cfb-1446-4a54-850e-dcc7698890d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:42.681312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d2e2cfb-1446-4a54-850e-dcc7698890d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:42.738533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d2e2cfb-1446-4a54-850e-dcc7698890d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:42.739016 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d2e2cfb-1446-4a54-850e-dcc7698890d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:42.739440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d2e2cfb-1446-4a54-850e-dcc7698890d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:42.739884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-4d2e2cfb-1446-4a54-850e-dcc7698890d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lazy-loading 'fault' on Instance uuid 483c7eb7-3827-403f-84f3-c7c65dcbe574 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 13:59:42.753993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-af1d1f89-54d5-4766-ad25-d886a4c9079b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:42.771088 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-af1d1f89-54d5-4766-ad25-d886a4c9079b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.295260 Sep 08 13:59:42.785953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 479/843] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:42 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:42.799650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4d2e2cfb-1446-4a54-850e-dcc7698890d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:42.808750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4d2e2cfb-1446-4a54-850e-dcc7698890d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1331 microversion: 2.1 time: 0.155580 Sep 08 13:59:42.810838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 365/844] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:42 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1331 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:43.234848 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7043644f-aaf2-45b0-8384-debacfe336c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:43.235687 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7043644f-aaf2-45b0-8384-debacfe336c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:43.242546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7043644f-aaf2-45b0-8384-debacfe336c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:43.243131 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7043644f-aaf2-45b0-8384-debacfe336c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:43.243645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7043644f-aaf2-45b0-8384-debacfe336c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:43.244444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7043644f-aaf2-45b0-8384-debacfe336c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:43.244979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7043644f-aaf2-45b0-8384-debacfe336c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:43.245643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7043644f-aaf2-45b0-8384-debacfe336c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:43.277489 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-07663fbb-9ac0-4751-800c-9de08a4b57ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:43.277489 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-07663fbb-9ac0-4751-800c-9de08a4b57ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:43.277489 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-07663fbb-9ac0-4751-800c-9de08a4b57ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:43.278981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-07663fbb-9ac0-4751-800c-9de08a4b57ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:43.278981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-07663fbb-9ac0-4751-800c-9de08a4b57ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:43.279741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-07663fbb-9ac0-4751-800c-9de08a4b57ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:43.280206 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-07663fbb-9ac0-4751-800c-9de08a4b57ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:43.280620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-07663fbb-9ac0-4751-800c-9de08a4b57ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:43.312608 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-07663fbb-9ac0-4751-800c-9de08a4b57ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052469 Sep 08 13:59:43.313364 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 366/845] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:43 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:43.722768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7043644f-aaf2-45b0-8384-debacfe336c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:43.731228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7043644f-aaf2-45b0-8384-debacfe336c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1735 microversion: 2.1 time: 0.500834 Sep 08 13:59:43.731811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 480/846] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:43 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1735 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:59:43.809928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-37cc1cc4-7627-403c-857e-afc10ff31bc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:43.810599 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-37cc1cc4-7627-403c-857e-afc10ff31bc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:43.816633 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-37cc1cc4-7627-403c-857e-afc10ff31bc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:43.817109 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-37cc1cc4-7627-403c-857e-afc10ff31bc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:43.817536 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-37cc1cc4-7627-403c-857e-afc10ff31bc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:43.818255 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-37cc1cc4-7627-403c-857e-afc10ff31bc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:43.818684 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-37cc1cc4-7627-403c-857e-afc10ff31bc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:43.819098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-37cc1cc4-7627-403c-857e-afc10ff31bc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:43.835963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7ccdf917-7703-457f-8d9c-e806ea68de80 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:43.836604 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7ccdf917-7703-457f-8d9c-e806ea68de80 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:43.843185 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7ccdf917-7703-457f-8d9c-e806ea68de80 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:43.843942 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7ccdf917-7703-457f-8d9c-e806ea68de80 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:43.844384 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7ccdf917-7703-457f-8d9c-e806ea68de80 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:43.845178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7ccdf917-7703-457f-8d9c-e806ea68de80 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:43.845573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7ccdf917-7703-457f-8d9c-e806ea68de80 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:43.847001 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7ccdf917-7703-457f-8d9c-e806ea68de80 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:43.885649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7ccdf917-7703-457f-8d9c-e806ea68de80 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:43.886514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7ccdf917-7703-457f-8d9c-e806ea68de80 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:43.887149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7ccdf917-7703-457f-8d9c-e806ea68de80 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:43.888249 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-7ccdf917-7703-457f-8d9c-e806ea68de80 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lazy-loading 'fault' on Instance uuid 483c7eb7-3827-403f-84f3-c7c65dcbe574 {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 13:59:43.946943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7ccdf917-7703-457f-8d9c-e806ea68de80 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:43.956327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7ccdf917-7703-457f-8d9c-e806ea68de80 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1331 microversion: 2.1 time: 0.124256 Sep 08 13:59:43.957247 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 481/847] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:43 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1331 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:59:43.968744 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-37cc1cc4-7627-403c-857e-afc10ff31bc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:43.976674 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-37cc1cc4-7627-403c-857e-afc10ff31bc4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.171260 Sep 08 13:59:43.977250 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 367/848] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:43 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:59:44.325707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ed1a0b50-134a-4c6b-b8dc-818775fd6705 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:44.328107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ed1a0b50-134a-4c6b-b8dc-818775fd6705 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:44.337989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed1a0b50-134a-4c6b-b8dc-818775fd6705 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:44.338740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed1a0b50-134a-4c6b-b8dc-818775fd6705 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:44.339164 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed1a0b50-134a-4c6b-b8dc-818775fd6705 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:44.340224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed1a0b50-134a-4c6b-b8dc-818775fd6705 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:44.340860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed1a0b50-134a-4c6b-b8dc-818775fd6705 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:44.341658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed1a0b50-134a-4c6b-b8dc-818775fd6705 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:44.375619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ed1a0b50-134a-4c6b-b8dc-818775fd6705 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054491 Sep 08 13:59:44.376265 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 482/849] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:44 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:44.757605 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-29d36b7f-20c0-4d2d-b39f-7dcd14f060ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:44.759044 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-29d36b7f-20c0-4d2d-b39f-7dcd14f060ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:44.766845 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29d36b7f-20c0-4d2d-b39f-7dcd14f060ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:44.767459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29d36b7f-20c0-4d2d-b39f-7dcd14f060ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:44.767967 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29d36b7f-20c0-4d2d-b39f-7dcd14f060ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:44.768721 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29d36b7f-20c0-4d2d-b39f-7dcd14f060ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:44.769238 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29d36b7f-20c0-4d2d-b39f-7dcd14f060ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:44.769718 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29d36b7f-20c0-4d2d-b39f-7dcd14f060ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:44.898424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-b678bbb4-9bc8-4ebd-a1ff-b47940067cdc req-5c95f0a8-cde1-4e38-b340-32ee6c8612b1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d7ae43e4-11c8-417c-be73-4f9fb5c42f2c", "tag": "48757fc0-370f-4fb9-a0d2-0688efed54e8"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:59:44.909788 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-b678bbb4-9bc8-4ebd-a1ff-b47940067cdc req-5c95f0a8-cde1-4e38-b340-32ee6c8612b1 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:44.910302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-b678bbb4-9bc8-4ebd-a1ff-b47940067cdc req-5c95f0a8-cde1-4e38-b340-32ee6c8612b1 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:44.911000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-b678bbb4-9bc8-4ebd-a1ff-b47940067cdc req-5c95f0a8-cde1-4e38-b340-32ee6c8612b1 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:44.918129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-29d36b7f-20c0-4d2d-b39f-7dcd14f060ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:44.924201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-b678bbb4-9bc8-4ebd-a1ff-b47940067cdc req-5c95f0a8-cde1-4e38-b340-32ee6c8612b1 service nova] Creating event network-changed:48757fc0-370f-4fb9-a0d2-0688efed54e8 for instance d7ae43e4-11c8-417c-be73-4f9fb5c42f2c on hv-cinder-85646 Sep 08 13:59:44.929183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-29d36b7f-20c0-4d2d-b39f-7dcd14f060ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1715 microversion: 2.1 time: 0.176894 Sep 08 13:59:44.929834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-b678bbb4-9bc8-4ebd-a1ff-b47940067cdc req-5c95f0a8-cde1-4e38-b340-32ee6c8612b1 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.111431 Sep 08 13:59:44.930468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 483/850] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:59:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:59:44.941912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 368/851] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:44 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1715 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:44.981265 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9bd81c9c-180d-4569-a976-9d359b73f3fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:44.982009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9bd81c9c-180d-4569-a976-9d359b73f3fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:44.988575 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bd81c9c-180d-4569-a976-9d359b73f3fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:44.989346 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bd81c9c-180d-4569-a976-9d359b73f3fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:44.990005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bd81c9c-180d-4569-a976-9d359b73f3fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:44.990928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bd81c9c-180d-4569-a976-9d359b73f3fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:44.997457 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1d34a5a1-45d9-464f-8886-dfc30618c7ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:44.997946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1d34a5a1-45d9-464f-8886-dfc30618c7ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:45.002361 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bd81c9c-180d-4569-a976-9d359b73f3fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:45.002361 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bd81c9c-180d-4569-a976-9d359b73f3fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:45.005869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d34a5a1-45d9-464f-8886-dfc30618c7ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:45.006400 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d34a5a1-45d9-464f-8886-dfc30618c7ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:45.007115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d34a5a1-45d9-464f-8886-dfc30618c7ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:45.007891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d34a5a1-45d9-464f-8886-dfc30618c7ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:45.008445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d34a5a1-45d9-464f-8886-dfc30618c7ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:45.008947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d34a5a1-45d9-464f-8886-dfc30618c7ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:45.041556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bd81c9c-180d-4569-a976-9d359b73f3fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:45.042671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bd81c9c-180d-4569-a976-9d359b73f3fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:45.043911 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9bd81c9c-180d-4569-a976-9d359b73f3fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:45.044618 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-9bd81c9c-180d-4569-a976-9d359b73f3fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lazy-loading 'fault' on Instance uuid 483c7eb7-3827-403f-84f3-c7c65dcbe574 {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 13:59:45.102911 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-1d34a5a1-45d9-464f-8886-dfc30618c7ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] HTTP exception thrown: Cannot 'detach_volume' instance 19db4aa2-f574-43fa-bbf8-c3042b243152 while it is in vm_state rescued Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1d34a5a1-45d9-464f-8886-dfc30618c7ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Request method failure captured: Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments/3d6a13a3-142c-49d7-9ef4-0c61f151b4c3 HTTP/1.1 Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPpGZzMisIeN0rinm76oerWeI1e2n1ZCDvu1_QxGMaqly5J66d47JWYR-nVcthn5vbK6XFUxRx7ULj4gJs_JU089dQK1DqQD0zha4-7jxb_89w0QMqZgtACCW34zdnpCyIfB12dgyAWUWv38AgePo6JrTClIfdFoSLdR5OVWnj2kmY Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 1346d4105dce4c6e928fcb11cde56854 Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-188566618 Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 13:59:45.105127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 13:59:45.107197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:59:45.107197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-ServerRescueNegativeTestJSON-188566618 Sep 08 13:59:45.107197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 1346d4105dce4c6e928fcb11cde56854 Sep 08 13:59:45.107197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-188566618 Sep 08 13:59:45.107197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-ServerRescueNegativeTestJSON-188566618-project Sep 08 13:59:45.107197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 13:59:45.107197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 13:59:45.107197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 792808dcbf894aa98c671cf43ac21b8b Sep 08 13:59:45.107197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-ServerRescueNegativeTestJSON-188566618-project Sep 08 13:59:45.107197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 13:59:45.107197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Cannot 'detach_volume' instance 19db4aa2-f574-43fa-bbf8-c3042b243152 while it is in vm_state rescued Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'server_id': '19db4aa2-f574-43fa-bbf8-c3042b243152', 'id': '3d6a13a3-142c-49d7-9ef4-0c61f151b4c3'} Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 19db4aa2-f574-43fa-bbf8-c3042b243152 in vm_state rescued. Cannot detach_volume while the instance is in this state. Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:59:45.115122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:59:45.116572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Sep 08 13:59:45.116572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 08 13:59:45.116572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 08 13:59:45.116572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 08 13:59:45.116572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 19db4aa2-f574-43fa-bbf8-c3042b243152 while it is in vm_state rescued Sep 08 13:59:45.116572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:59:45.116572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:59:45.116572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:59:45.116572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:59:45.116572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:59:45.116572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:59:45.116572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:59:45.116572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:59:45.116572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 19db4aa2-f574-43fa-bbf8-c3042b243152 while it is in vm_state rescued Sep 08 13:59:45.116572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 13:59:45.116572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1d34a5a1-45d9-464f-8886-dfc30618c7ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Returning 409 to user: Cannot 'detach_volume' instance 19db4aa2-f574-43fa-bbf8-c3042b243152 while it is in vm_state rescued {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:59:45.117596 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1d34a5a1-45d9-464f-8886-dfc30618c7ed tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments/3d6a13a3-142c-49d7-9ef4-0c61f151b4c3" status: 409 len: 152 microversion: 2.1 time: 0.114371 Sep 08 13:59:45.117676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9bd81c9c-180d-4569-a976-9d359b73f3fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:45.117769 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 369/852] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 13:59:44 2022] DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments/3d6a13a3-142c-49d7-9ef4-0c61f151b4c3 => generated 152 bytes in 116 msecs (HTTP/1.1 409) 9 headers in 378 bytes (2 switches on core 0) Sep 08 13:59:45.117769 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4d1c3e3c-add6-4cd4-b415-057f2981c600 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:45.117926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9bd81c9c-180d-4569-a976-9d359b73f3fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1331 microversion: 2.1 time: 0.138996 Sep 08 13:59:45.117926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 484/853] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:44 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1331 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:45.118067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4d1c3e3c-add6-4cd4-b415-057f2981c600 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:45.120687 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e7405176-69f6-4c61-94c3-ea108bc606ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:59:45.121221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e7405176-69f6-4c61-94c3-ea108bc606ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:45.125776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d1c3e3c-add6-4cd4-b415-057f2981c600 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:45.126495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d1c3e3c-add6-4cd4-b415-057f2981c600 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:45.127040 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d1c3e3c-add6-4cd4-b415-057f2981c600 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:45.127848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d1c3e3c-add6-4cd4-b415-057f2981c600 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:45.127975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e7405176-69f6-4c61-94c3-ea108bc606ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:45.128424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e7405176-69f6-4c61-94c3-ea108bc606ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:45.128564 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d1c3e3c-add6-4cd4-b415-057f2981c600 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:45.128871 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e7405176-69f6-4c61-94c3-ea108bc606ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:45.129185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d1c3e3c-add6-4cd4-b415-057f2981c600 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:45.129570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e7405176-69f6-4c61-94c3-ea108bc606ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:45.130029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e7405176-69f6-4c61-94c3-ea108bc606ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:45.130494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e7405176-69f6-4c61-94c3-ea108bc606ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:45.329812 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4d1c3e3c-add6-4cd4-b415-057f2981c600 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:45.338430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4d1c3e3c-add6-4cd4-b415-057f2981c600 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1260 microversion: 2.1 time: 0.229111 Sep 08 13:59:45.354284 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 370/854] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:45 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1260 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:45.395826 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ccfc1315-ff7c-487d-8993-855668808d64 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:45.400167 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e7405176-69f6-4c61-94c3-ea108bc606ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "POST /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/action" status: 202 len: 0 microversion: 2.1 time: 0.282921 Sep 08 13:59:45.400654 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ccfc1315-ff7c-487d-8993-855668808d64 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:45.400794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 485/855] 10.222.0.25 () {60 vars in 1278 bytes} [Thu Sep 8 13:59:45 2022] POST /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/action => generated 0 bytes in 284 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 13:59:45.410079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d9263c13-e5c6-4661-a84f-7fd862f66825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:45.410879 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d9263c13-e5c6-4661-a84f-7fd862f66825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:45.411016 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccfc1315-ff7c-487d-8993-855668808d64 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:45.412305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccfc1315-ff7c-487d-8993-855668808d64 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:45.412305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccfc1315-ff7c-487d-8993-855668808d64 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:45.412907 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccfc1315-ff7c-487d-8993-855668808d64 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:45.413983 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccfc1315-ff7c-487d-8993-855668808d64 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:45.414555 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccfc1315-ff7c-487d-8993-855668808d64 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:45.425704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d9263c13-e5c6-4661-a84f-7fd862f66825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:45.426261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d9263c13-e5c6-4661-a84f-7fd862f66825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:45.428236 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d9263c13-e5c6-4661-a84f-7fd862f66825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:45.429656 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d9263c13-e5c6-4661-a84f-7fd862f66825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:45.430342 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d9263c13-e5c6-4661-a84f-7fd862f66825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:45.444484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d9263c13-e5c6-4661-a84f-7fd862f66825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:45.493200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ccfc1315-ff7c-487d-8993-855668808d64 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.108097 Sep 08 13:59:45.494007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 371/856] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:45 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:45.647907 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d9263c13-e5c6-4661-a84f-7fd862f66825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:45.659263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d9263c13-e5c6-4661-a84f-7fd862f66825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1723 microversion: 2.1 time: 0.253269 Sep 08 13:59:45.660126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 486/857] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:45 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1723 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:59:46.140163 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8d8a13e8-3032-4e77-a6bf-81cb68be34e7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:46.140768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8d8a13e8-3032-4e77-a6bf-81cb68be34e7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:46.147363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d8a13e8-3032-4e77-a6bf-81cb68be34e7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:46.147853 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d8a13e8-3032-4e77-a6bf-81cb68be34e7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:46.148287 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d8a13e8-3032-4e77-a6bf-81cb68be34e7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:46.148983 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d8a13e8-3032-4e77-a6bf-81cb68be34e7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:46.149434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d8a13e8-3032-4e77-a6bf-81cb68be34e7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:46.149855 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d8a13e8-3032-4e77-a6bf-81cb68be34e7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:46.179172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:46.185389 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d8a13e8-3032-4e77-a6bf-81cb68be34e7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:46.185946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d8a13e8-3032-4e77-a6bf-81cb68be34e7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:46.186939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d8a13e8-3032-4e77-a6bf-81cb68be34e7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:46.189728 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-8d8a13e8-3032-4e77-a6bf-81cb68be34e7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lazy-loading 'fault' on Instance uuid 483c7eb7-3827-403f-84f3-c7c65dcbe574 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 13:59:46.240918 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8d8a13e8-3032-4e77-a6bf-81cb68be34e7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:46.249393 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8d8a13e8-3032-4e77-a6bf-81cb68be34e7 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 200 len: 1331 microversion: 2.1 time: 0.113933 Sep 08 13:59:46.249959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 372/858] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:46 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 1331 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:46.360848 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d61c6ca8-fa7e-4b07-aef1-1e9bb2e7e685 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:46.361471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d61c6ca8-fa7e-4b07-aef1-1e9bb2e7e685 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:46.368395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d61c6ca8-fa7e-4b07-aef1-1e9bb2e7e685 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:46.369179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d61c6ca8-fa7e-4b07-aef1-1e9bb2e7e685 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:46.369329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d61c6ca8-fa7e-4b07-aef1-1e9bb2e7e685 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:46.370004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d61c6ca8-fa7e-4b07-aef1-1e9bb2e7e685 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:46.370519 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d61c6ca8-fa7e-4b07-aef1-1e9bb2e7e685 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:46.370948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d61c6ca8-fa7e-4b07-aef1-1e9bb2e7e685 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:46.508604 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f4a0df13-35f2-422f-b4d7-e37d91fbf700 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:46.512585 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f4a0df13-35f2-422f-b4d7-e37d91fbf700 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:46.528379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f4a0df13-35f2-422f-b4d7-e37d91fbf700 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:46.528869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f4a0df13-35f2-422f-b4d7-e37d91fbf700 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:46.529715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f4a0df13-35f2-422f-b4d7-e37d91fbf700 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:46.530484 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f4a0df13-35f2-422f-b4d7-e37d91fbf700 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:46.530940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f4a0df13-35f2-422f-b4d7-e37d91fbf700 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:46.531365 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f4a0df13-35f2-422f-b4d7-e37d91fbf700 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:46.547226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d61c6ca8-fa7e-4b07-aef1-1e9bb2e7e685 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:46.558363 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d61c6ca8-fa7e-4b07-aef1-1e9bb2e7e685 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1445 microversion: 2.1 time: 0.201078 Sep 08 13:59:46.560292 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 487/859] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:46 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1445 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:46.571462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f4a0df13-35f2-422f-b4d7-e37d91fbf700 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.067271 Sep 08 13:59:46.572030 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 373/860] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:46 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:46.582516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4ffb5d68-6552-4e56-8d2a-4a49f2f12782 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:46.583208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4ffb5d68-6552-4e56-8d2a-4a49f2f12782 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:46.589536 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ffb5d68-6552-4e56-8d2a-4a49f2f12782 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:46.590401 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ffb5d68-6552-4e56-8d2a-4a49f2f12782 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:46.590848 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ffb5d68-6552-4e56-8d2a-4a49f2f12782 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:46.591545 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ffb5d68-6552-4e56-8d2a-4a49f2f12782 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:46.592004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ffb5d68-6552-4e56-8d2a-4a49f2f12782 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:46.592426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ffb5d68-6552-4e56-8d2a-4a49f2f12782 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:46.683050 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3d156f47-2cf4-4c06-a0c4-55c4e8c2065c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:46.683504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3d156f47-2cf4-4c06-a0c4-55c4e8c2065c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:46.690111 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d156f47-2cf4-4c06-a0c4-55c4e8c2065c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:46.693670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d156f47-2cf4-4c06-a0c4-55c4e8c2065c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:46.693670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d156f47-2cf4-4c06-a0c4-55c4e8c2065c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:46.693670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d156f47-2cf4-4c06-a0c4-55c4e8c2065c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:46.693670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d156f47-2cf4-4c06-a0c4-55c4e8c2065c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:46.694338 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d156f47-2cf4-4c06-a0c4-55c4e8c2065c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:46.748054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4ffb5d68-6552-4e56-8d2a-4a49f2f12782 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:46.756124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4ffb5d68-6552-4e56-8d2a-4a49f2f12782 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1445 microversion: 2.1 time: 0.177445 Sep 08 13:59:46.756613 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 488/861] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:46 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1445 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:47.004077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3d156f47-2cf4-4c06-a0c4-55c4e8c2065c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:47.014431 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3d156f47-2cf4-4c06-a0c4-55c4e8c2065c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1723 microversion: 2.1 time: 0.335330 Sep 08 13:59:47.026802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 374/862] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:46 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1723 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:59:47.275802 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-74e959d2-b773-4012-9e44-bec969110249 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:47.278037 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-74e959d2-b773-4012-9e44-bec969110249 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:47.287292 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-74e959d2-b773-4012-9e44-bec969110249 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:47.287292 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-74e959d2-b773-4012-9e44-bec969110249 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:47.287292 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-74e959d2-b773-4012-9e44-bec969110249 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:47.287865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-74e959d2-b773-4012-9e44-bec969110249 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:47.288432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-74e959d2-b773-4012-9e44-bec969110249 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:47.289058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-74e959d2-b773-4012-9e44-bec969110249 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:47.305655 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-74e959d2-b773-4012-9e44-bec969110249 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 could not be found. Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-74e959d2-b773-4012-9e44-bec969110249 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 HTTP/1.1 Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 13:59:47.307520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 13:59:47.309671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 13:59:47.309671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 13:59:47.309671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 13:59:47.309671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 13:59:47.309671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 13:59:47.309671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 13:59:47.309671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 13:59:47.309671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 13:59:47.309671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 13:59:47.309671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 13:59:47.309671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 could not be found. Sep 08 13:59:47.309671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': '483c7eb7-3827-403f-84f3-c7c65dcbe574'} Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 could not be found. Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 13:59:47.310628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 13:59:47.312829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 13:59:47.312829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 13:59:47.312829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 13:59:47.312829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 13:59:47.312829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 13:59:47.312829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 13:59:47.312829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 13:59:47.312829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 could not be found. Sep 08 13:59:47.312829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:59:47.312829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 13:59:47.312829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:59:47.312829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 13:59:47.312829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 13:59:47.312829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 13:59:47.312829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 13:59:47.312829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 13:59:47.312829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 could not be found. Sep 08 13:59:47.312829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 13:59:47.317411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-74e959d2-b773-4012-9e44-bec969110249 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 13:59:47.317411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-74e959d2-b773-4012-9e44-bec969110249 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 404 len: 111 microversion: 2.1 time: 0.043392 Sep 08 13:59:47.317411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 489/863] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:47 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 13:59:47.324633 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7256f3af-fe12-4047-b607-a3430d81302e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '.version_select at 0x7fce020895e0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:47.455254 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7256f3af-fe12-4047-b607-a3430d81302e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-494104329" status: 202 len: 0 microversion: 2.1 time: 0.134356 Sep 08 13:59:47.457485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 375/864] 10.222.0.25 () {58 vars in 1218 bytes} [Thu Sep 8 13:59:47 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-494104329 => generated 0 bytes in 137 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 13:59:47.586515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-007a6ab7-1afb-4301-9766-b528ce8ba9cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:47.590422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-007a6ab7-1afb-4301-9766-b528ce8ba9cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:47.597347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-007a6ab7-1afb-4301-9766-b528ce8ba9cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:47.598089 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-007a6ab7-1afb-4301-9766-b528ce8ba9cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:47.599391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-007a6ab7-1afb-4301-9766-b528ce8ba9cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:47.600270 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-007a6ab7-1afb-4301-9766-b528ce8ba9cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:47.600886 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-007a6ab7-1afb-4301-9766-b528ce8ba9cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:47.601472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-007a6ab7-1afb-4301-9766-b528ce8ba9cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:47.606907 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:47.647101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-007a6ab7-1afb-4301-9766-b528ce8ba9cf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.065358 Sep 08 13:59:47.648930 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 490/865] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:47 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:47.873903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-19d56183-c403-46d8-b5a0-1c0945c79b09 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-656088964"}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:59:47.881931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-19d56183-c403-46d8-b5a0-1c0945c79b09 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:47.897370 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19d56183-c403-46d8-b5a0-1c0945c79b09 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:47.904413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19d56183-c403-46d8-b5a0-1c0945c79b09 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:47.904967 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19d56183-c403-46d8-b5a0-1c0945c79b09 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:47.905864 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19d56183-c403-46d8-b5a0-1c0945c79b09 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:47.907637 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19d56183-c403-46d8-b5a0-1c0945c79b09 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:47.908816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19d56183-c403-46d8-b5a0-1c0945c79b09 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:48.042725 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bd809e0e-aa61-400d-849e-6151e112ea49 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:48.043571 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bd809e0e-aa61-400d-849e-6151e112ea49 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:48.053060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd809e0e-aa61-400d-849e-6151e112ea49 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:48.054558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd809e0e-aa61-400d-849e-6151e112ea49 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:48.059547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd809e0e-aa61-400d-849e-6151e112ea49 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:48.059547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd809e0e-aa61-400d-849e-6151e112ea49 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:48.059547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd809e0e-aa61-400d-849e-6151e112ea49 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:48.059547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd809e0e-aa61-400d-849e-6151e112ea49 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:48.223750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.network.security_group_api [None req-19d56183-c403-46d8-b5a0-1c0945c79b09 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Adding security group a479c883-ffb8-4546-b5d7-d2563c548f66 to port 48757fc0-370f-4fb9-a0d2-0688efed54e8 Sep 08 13:59:48.594557 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bd809e0e-aa61-400d-849e-6151e112ea49 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:48.607163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bd809e0e-aa61-400d-849e-6151e112ea49 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1723 microversion: 2.1 time: 0.572060 Sep 08 13:59:48.607887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 491/866] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:48 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1723 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:48.633060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-19d56183-c403-46d8-b5a0-1c0945c79b09 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "POST /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/action" status: 202 len: 0 microversion: 2.1 time: 0.768968 Sep 08 13:59:48.643260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 376/867] 10.222.0.25 () {60 vars in 1278 bytes} [Thu Sep 8 13:59:47 2022] POST /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/action => generated 0 bytes in 780 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 13:59:48.648548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-acae82a5-5358-49d5-b328-0bd71545daeb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:48.649237 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-acae82a5-5358-49d5-b328-0bd71545daeb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:48.659262 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-acae82a5-5358-49d5-b328-0bd71545daeb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:48.660204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-acae82a5-5358-49d5-b328-0bd71545daeb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:48.662310 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-acae82a5-5358-49d5-b328-0bd71545daeb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:48.663391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-acae82a5-5358-49d5-b328-0bd71545daeb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:48.663878 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-008ba170-d437-4aba-9c9b-85621733be08 req-78d7f22d-d62f-4a0d-8789-36b13f3cc06e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d7ae43e4-11c8-417c-be73-4f9fb5c42f2c", "tag": "48757fc0-370f-4fb9-a0d2-0688efed54e8"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:59:48.664277 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-acae82a5-5358-49d5-b328-0bd71545daeb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:48.664921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-acae82a5-5358-49d5-b328-0bd71545daeb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:48.677958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-008ba170-d437-4aba-9c9b-85621733be08 req-78d7f22d-d62f-4a0d-8789-36b13f3cc06e service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:48.679115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-008ba170-d437-4aba-9c9b-85621733be08 req-78d7f22d-d62f-4a0d-8789-36b13f3cc06e service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:48.679989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-008ba170-d437-4aba-9c9b-85621733be08 req-78d7f22d-d62f-4a0d-8789-36b13f3cc06e service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:48.697471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-008ba170-d437-4aba-9c9b-85621733be08 req-78d7f22d-d62f-4a0d-8789-36b13f3cc06e service nova] Creating event network-changed:48757fc0-370f-4fb9-a0d2-0688efed54e8 for instance d7ae43e4-11c8-417c-be73-4f9fb5c42f2c on hv-cinder-85646 Sep 08 13:59:48.706841 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-008ba170-d437-4aba-9c9b-85621733be08 req-78d7f22d-d62f-4a0d-8789-36b13f3cc06e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.049117 Sep 08 13:59:48.709169 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 377/868] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:59:48 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:59:48.719560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-012d121b-f460-48f1-ace8-2403f913380d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:48.722837 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-012d121b-f460-48f1-ace8-2403f913380d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:48.732480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-012d121b-f460-48f1-ace8-2403f913380d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:48.733269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-012d121b-f460-48f1-ace8-2403f913380d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:48.734620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-012d121b-f460-48f1-ace8-2403f913380d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:48.735746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-012d121b-f460-48f1-ace8-2403f913380d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:48.736729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-012d121b-f460-48f1-ace8-2403f913380d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:48.738082 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-012d121b-f460-48f1-ace8-2403f913380d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:48.778274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-012d121b-f460-48f1-ace8-2403f913380d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.066391 Sep 08 13:59:48.784197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 378/869] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:48 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:48.992257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-acae82a5-5358-49d5-b328-0bd71545daeb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:49.001882 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-acae82a5-5358-49d5-b328-0bd71545daeb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1757 microversion: 2.1 time: 0.357503 Sep 08 13:59:49.002436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 492/870] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:48 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1757 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:59:49.315429 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-43b6efa1-c1bd-462d-be83-aa8d0f0d8e17 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Action: 'create', calling method: .version_select at 0x7fce02089940>, body: {"keypair": {"name": "tempest-keypair-997401276", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK3FpNDT8zdnuIvebCkk1xYS6dmlQJXFJdBbABWwenLYVxOfJZSusQRhBCZ0V62W3UxceGIBg9DMh14lC+hN8YAetaXwGYCpKjhEKhmSsvkY5tCSVvPUSTuzEX/PSsJ7Fw=="}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:59:49.331717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-43b6efa1-c1bd-462d-be83-aa8d0f0d8e17 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Getting quotas for project a4e0fba6cab34c429c73fcfac7ebfd94. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:59:49.336072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-43b6efa1-c1bd-462d-be83-aa8d0f0d8e17 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Getting quotas for user eca89f435299415c8448931e73f5b0fc and project a4e0fba6cab34c429c73fcfac7ebfd94. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:59:49.437480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-43b6efa1-c1bd-462d-be83-aa8d0f0d8e17 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.126540 Sep 08 13:59:49.438092 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 379/871] 10.222.0.25 () {60 vars in 1159 bytes} [Thu Sep 8 13:59:49 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:59:49.632201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fdac895b-ba44-4f09-bd0e-6750962624a9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:49.633984 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fdac895b-ba44-4f09-bd0e-6750962624a9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:49.640846 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:49.644253 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fdac895b-ba44-4f09-bd0e-6750962624a9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:49.644856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fdac895b-ba44-4f09-bd0e-6750962624a9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:49.645278 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fdac895b-ba44-4f09-bd0e-6750962624a9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:49.645998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fdac895b-ba44-4f09-bd0e-6750962624a9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:49.646587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fdac895b-ba44-4f09-bd0e-6750962624a9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:49.647289 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fdac895b-ba44-4f09-bd0e-6750962624a9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:49.792454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-41f6f686-18a4-4fc0-9811-c8ae3919cbdf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:49.795046 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-41f6f686-18a4-4fc0-9811-c8ae3919cbdf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:49.811793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-41f6f686-18a4-4fc0-9811-c8ae3919cbdf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:49.812364 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-41f6f686-18a4-4fc0-9811-c8ae3919cbdf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:49.812880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-41f6f686-18a4-4fc0-9811-c8ae3919cbdf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:49.813708 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-41f6f686-18a4-4fc0-9811-c8ae3919cbdf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:49.814272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-41f6f686-18a4-4fc0-9811-c8ae3919cbdf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:49.814826 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-41f6f686-18a4-4fc0-9811-c8ae3919cbdf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:49.848578 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-41f6f686-18a4-4fc0-9811-c8ae3919cbdf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061103 Sep 08 13:59:49.849151 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 380/872] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:49 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:59:49.871416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fdac895b-ba44-4f09-bd0e-6750962624a9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:49.879911 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fdac895b-ba44-4f09-bd0e-6750962624a9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1723 microversion: 2.1 time: 0.251681 Sep 08 13:59:49.882399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 493/873] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:49 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1723 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:50.157576 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-162e1155-6daa-4cba-950c-b2c17258b15d req-32029a6a-03c1-4f3b-85d5-140fc143750c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d1062abf-7525-48c5-a9cb-91da3855fd1b", "tag": "2d04f3d2-99c9-4712-bc13-6ebdffcb949f"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:59:50.168842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-162e1155-6daa-4cba-950c-b2c17258b15d req-32029a6a-03c1-4f3b-85d5-140fc143750c service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:50.169330 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-162e1155-6daa-4cba-950c-b2c17258b15d req-32029a6a-03c1-4f3b-85d5-140fc143750c service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:50.169761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-162e1155-6daa-4cba-950c-b2c17258b15d req-32029a6a-03c1-4f3b-85d5-140fc143750c service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:50.181522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-162e1155-6daa-4cba-950c-b2c17258b15d req-32029a6a-03c1-4f3b-85d5-140fc143750c service nova] Creating event network-changed:2d04f3d2-99c9-4712-bc13-6ebdffcb949f for instance d1062abf-7525-48c5-a9cb-91da3855fd1b on hv-cinder-85646 Sep 08 13:59:50.187133 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-162e1155-6daa-4cba-950c-b2c17258b15d req-32029a6a-03c1-4f3b-85d5-140fc143750c service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035309 Sep 08 13:59:50.188080 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 381/874] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:59:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:59:50.300688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6f28232e-77ae-4a16-a4aa-a4d0242d9aae tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:59:50.303721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6f28232e-77ae-4a16-a4aa-a4d0242d9aae tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:50.309677 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f28232e-77ae-4a16-a4aa-a4d0242d9aae tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:50.310209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f28232e-77ae-4a16-a4aa-a4d0242d9aae tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:50.310826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f28232e-77ae-4a16-a4aa-a4d0242d9aae tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:50.311524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f28232e-77ae-4a16-a4aa-a4d0242d9aae tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:50.311979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f28232e-77ae-4a16-a4aa-a4d0242d9aae tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:50.312400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f28232e-77ae-4a16-a4aa-a4d0242d9aae tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:50.611872 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6f28232e-77ae-4a16-a4aa-a4d0242d9aae tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "POST /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/action" status: 202 len: 0 microversion: 2.1 time: 0.314888 Sep 08 13:59:50.615858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 494/875] 10.222.0.25 () {60 vars in 1278 bytes} [Thu Sep 8 13:59:50 2022] POST /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/action => generated 0 bytes in 317 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 13:59:50.622523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ae64202d-cafe-46a5-899c-170b5ed4b8c1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:50.623255 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ae64202d-cafe-46a5-899c-170b5ed4b8c1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:50.629717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ae64202d-cafe-46a5-899c-170b5ed4b8c1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:50.630384 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ae64202d-cafe-46a5-899c-170b5ed4b8c1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:50.631157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ae64202d-cafe-46a5-899c-170b5ed4b8c1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:50.631872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ae64202d-cafe-46a5-899c-170b5ed4b8c1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:50.632332 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ae64202d-cafe-46a5-899c-170b5ed4b8c1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:50.632857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ae64202d-cafe-46a5-899c-170b5ed4b8c1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:50.791331 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ae64202d-cafe-46a5-899c-170b5ed4b8c1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:50.801495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ae64202d-cafe-46a5-899c-170b5ed4b8c1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1749 microversion: 2.1 time: 0.183751 Sep 08 13:59:50.803118 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 382/876] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:50 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1749 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:50.861521 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8edf9355-f2fd-47d1-95e7-b26e1c3be577 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:50.864641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8edf9355-f2fd-47d1-95e7-b26e1c3be577 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:50.871929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8edf9355-f2fd-47d1-95e7-b26e1c3be577 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:50.872817 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8edf9355-f2fd-47d1-95e7-b26e1c3be577 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:50.873491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8edf9355-f2fd-47d1-95e7-b26e1c3be577 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:50.874317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8edf9355-f2fd-47d1-95e7-b26e1c3be577 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:50.874998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8edf9355-f2fd-47d1-95e7-b26e1c3be577 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:50.875637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8edf9355-f2fd-47d1-95e7-b26e1c3be577 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:50.906708 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3fff8ac9-ae7b-422c-947a-a682c243bf87 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:50.907328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3fff8ac9-ae7b-422c-947a-a682c243bf87 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:50.912768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8edf9355-f2fd-47d1-95e7-b26e1c3be577 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055439 Sep 08 13:59:50.913419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3fff8ac9-ae7b-422c-947a-a682c243bf87 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:50.913610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 495/877] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:50 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:50.914672 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3fff8ac9-ae7b-422c-947a-a682c243bf87 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:50.915148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3fff8ac9-ae7b-422c-947a-a682c243bf87 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:50.915885 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3fff8ac9-ae7b-422c-947a-a682c243bf87 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:50.916530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3fff8ac9-ae7b-422c-947a-a682c243bf87 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:50.917130 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3fff8ac9-ae7b-422c-947a-a682c243bf87 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:51.233892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3fff8ac9-ae7b-422c-947a-a682c243bf87 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:51.242128 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3fff8ac9-ae7b-422c-947a-a682c243bf87 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 200 len: 1729 microversion: 2.1 time: 0.340990 Sep 08 13:59:51.242735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 383/878] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:50 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 1729 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:51.270620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5aa0eb1b-24aa-44bc-8b57-5c7dfc79f17c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:51.271420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5aa0eb1b-24aa-44bc-8b57-5c7dfc79f17c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:51.279650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aa0eb1b-24aa-44bc-8b57-5c7dfc79f17c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:51.280512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aa0eb1b-24aa-44bc-8b57-5c7dfc79f17c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:51.281153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aa0eb1b-24aa-44bc-8b57-5c7dfc79f17c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:51.282123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aa0eb1b-24aa-44bc-8b57-5c7dfc79f17c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:51.283952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aa0eb1b-24aa-44bc-8b57-5c7dfc79f17c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:51.284639 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aa0eb1b-24aa-44bc-8b57-5c7dfc79f17c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:51.431519 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5aa0eb1b-24aa-44bc-8b57-5c7dfc79f17c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.165287 Sep 08 13:59:51.435135 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 496/879] 10.222.0.25 () {58 vars in 1275 bytes} [Thu Sep 8 13:59:51 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-interface => generated 284 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:59:51.827486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-08c50c2f-790b-465f-abde-37f9a1820bd5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:51.828628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-08c50c2f-790b-465f-abde-37f9a1820bd5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:51.836888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08c50c2f-790b-465f-abde-37f9a1820bd5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:51.837539 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08c50c2f-790b-465f-abde-37f9a1820bd5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:51.838389 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08c50c2f-790b-465f-abde-37f9a1820bd5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:51.839204 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08c50c2f-790b-465f-abde-37f9a1820bd5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:51.839782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08c50c2f-790b-465f-abde-37f9a1820bd5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:51.840296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08c50c2f-790b-465f-abde-37f9a1820bd5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:51.862584 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:51.925878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ee3233e6-4b9b-4dc5-af55-cad36599eaa4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:51.928588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ee3233e6-4b9b-4dc5-af55-cad36599eaa4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:51.935392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ee3233e6-4b9b-4dc5-af55-cad36599eaa4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:51.936103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ee3233e6-4b9b-4dc5-af55-cad36599eaa4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:51.936777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ee3233e6-4b9b-4dc5-af55-cad36599eaa4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:51.937490 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ee3233e6-4b9b-4dc5-af55-cad36599eaa4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:51.937981 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ee3233e6-4b9b-4dc5-af55-cad36599eaa4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:51.938513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ee3233e6-4b9b-4dc5-af55-cad36599eaa4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:51.974788 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ee3233e6-4b9b-4dc5-af55-cad36599eaa4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053485 Sep 08 13:59:51.975891 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 497/880] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:51 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:59:52.093849 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-08c50c2f-790b-465f-abde-37f9a1820bd5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:52.103538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-08c50c2f-790b-465f-abde-37f9a1820bd5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1749 microversion: 2.1 time: 0.280784 Sep 08 13:59:52.104894 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 384/881] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:51 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1749 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:52.562822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a578a46f-9cf0-4dc7-8b5a-35bf1f2dd21a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-533441044", "imageRef": "20b30425-92e5-4d78-8502-45557526ea49", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "a265cc60-700e-4951-9295-70d02eb6ee91"}], "security_groups": [{"name": "tempest-securitygroup--878734267"}], "key_name": "tempest-keypair-997401276", "user_data": "}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:59:52.724032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.network.neutron [None req-a578a46f-9cf0-4dc7-8b5a-35bf1f2dd21a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] validate_networks() for [('a265cc60-700e-4951-9295-70d02eb6ee91', None, None, None, None, None)] {{(pid=127601) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 13:59:52.988334 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-55beadf6-329b-4d7d-a4db-ba02712ce091 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:52.990559 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-55beadf6-329b-4d7d-a4db-ba02712ce091 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:52.997694 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-55beadf6-329b-4d7d-a4db-ba02712ce091 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:52.998206 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-55beadf6-329b-4d7d-a4db-ba02712ce091 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:52.998642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-55beadf6-329b-4d7d-a4db-ba02712ce091 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:52.999327 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-55beadf6-329b-4d7d-a4db-ba02712ce091 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:52.999770 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-55beadf6-329b-4d7d-a4db-ba02712ce091 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:53.000180 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-55beadf6-329b-4d7d-a4db-ba02712ce091 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:53.036018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-55beadf6-329b-4d7d-a4db-ba02712ce091 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051847 Sep 08 13:59:53.036776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 385/882] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:52 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:59:53.109875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-c1e883be-a6ed-4853-8c7f-5e357db1c2b1 req-ddb26e66-50a6-4949-bdc6-faea6e2e6019 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "19db4aa2-f574-43fa-bbf8-c3042b243152", "tag": "2ebf3a50-af15-4ecc-820b-87ad0a9ff652"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:59:53.120646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-c1e883be-a6ed-4853-8c7f-5e357db1c2b1 req-ddb26e66-50a6-4949-bdc6-faea6e2e6019 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:53.121119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-c1e883be-a6ed-4853-8c7f-5e357db1c2b1 req-ddb26e66-50a6-4949-bdc6-faea6e2e6019 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:53.121554 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-c1e883be-a6ed-4853-8c7f-5e357db1c2b1 req-ddb26e66-50a6-4949-bdc6-faea6e2e6019 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:53.135494 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-c1e883be-a6ed-4853-8c7f-5e357db1c2b1 req-ddb26e66-50a6-4949-bdc6-faea6e2e6019 service nova] Creating event network-changed:2ebf3a50-af15-4ecc-820b-87ad0a9ff652 for instance 19db4aa2-f574-43fa-bbf8-c3042b243152 on hv-cinder-85646 Sep 08 13:59:53.140413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-c1e883be-a6ed-4853-8c7f-5e357db1c2b1 req-ddb26e66-50a6-4949-bdc6-faea6e2e6019 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034494 Sep 08 13:59:53.141020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 386/883] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:59:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 13:59:53.146242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fc816a71-690e-4d53-95a4-af892cb5f6f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:53.146933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fc816a71-690e-4d53-95a4-af892cb5f6f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:53.153490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc816a71-690e-4d53-95a4-af892cb5f6f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:53.153983 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc816a71-690e-4d53-95a4-af892cb5f6f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:53.154440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc816a71-690e-4d53-95a4-af892cb5f6f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:53.155621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc816a71-690e-4d53-95a4-af892cb5f6f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:53.155621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc816a71-690e-4d53-95a4-af892cb5f6f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:53.156003 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc816a71-690e-4d53-95a4-af892cb5f6f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:53.315686 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-a578a46f-9cf0-4dc7-8b5a-35bf1f2dd21a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Flavor limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 13:59:53.316252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-a578a46f-9cf0-4dc7-8b5a-35bf1f2dd21a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Image limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 13:59:53.316910 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-a578a46f-9cf0-4dc7-8b5a-35bf1f2dd21a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Flavor pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 13:59:53.317324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-a578a46f-9cf0-4dc7-8b5a-35bf1f2dd21a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Image pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 13:59:53.317742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-a578a46f-9cf0-4dc7-8b5a-35bf1f2dd21a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 13:59:53.327329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a578a46f-9cf0-4dc7-8b5a-35bf1f2dd21a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:53.327821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a578a46f-9cf0-4dc7-8b5a-35bf1f2dd21a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:53.328330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a578a46f-9cf0-4dc7-8b5a-35bf1f2dd21a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:53.329055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a578a46f-9cf0-4dc7-8b5a-35bf1f2dd21a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:53.329510 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a578a46f-9cf0-4dc7-8b5a-35bf1f2dd21a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:53.329931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a578a46f-9cf0-4dc7-8b5a-35bf1f2dd21a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:53.349125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-a578a46f-9cf0-4dc7-8b5a-35bf1f2dd21a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Getting quotas for project a4e0fba6cab34c429c73fcfac7ebfd94. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 13:59:53.353223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-a578a46f-9cf0-4dc7-8b5a-35bf1f2dd21a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Getting quotas for user eca89f435299415c8448931e73f5b0fc and project a4e0fba6cab34c429c73fcfac7ebfd94. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 13:59:53.365211 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a578a46f-9cf0-4dc7-8b5a-35bf1f2dd21a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Going to run 1 instances... {{(pid=127601) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 13:59:53.367799 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a578a46f-9cf0-4dc7-8b5a-35bf1f2dd21a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='20b30425-92e5-4d78-8502-45557526ea49',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127601) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 13:59:53.388992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-fc816a71-690e-4d53-95a4-af892cb5f6f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:53.398033 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fc816a71-690e-4d53-95a4-af892cb5f6f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1754 microversion: 2.1 time: 0.255592 Sep 08 13:59:53.398831 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 387/884] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:53 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1754 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:53.513420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a578a46f-9cf0-4dc7-8b5a-35bf1f2dd21a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.954822 Sep 08 13:59:53.514372 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 498/885] 10.222.0.25 () {60 vars in 1147 bytes} [Thu Sep 8 13:59:52 2022] POST /compute/v2.1/servers => generated 401 bytes in 957 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 08 13:59:53.527792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-000fe668-2eed-497b-9ff7-cc0dd4605658 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:53.528401 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-000fe668-2eed-497b-9ff7-cc0dd4605658 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:53.617829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-000fe668-2eed-497b-9ff7-cc0dd4605658 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:53.625254 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-000fe668-2eed-497b-9ff7-cc0dd4605658 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 200 len: 1240 microversion: 2.1 time: 0.102226 Sep 08 13:59:53.625829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 388/886] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:53 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 1240 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:54.048857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-57d805c3-c697-4161-b6d7-e7cf3cd640ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:54.051083 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-57d805c3-c697-4161-b6d7-e7cf3cd640ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:54.059331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57d805c3-c697-4161-b6d7-e7cf3cd640ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:54.059816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57d805c3-c697-4161-b6d7-e7cf3cd640ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:54.060253 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57d805c3-c697-4161-b6d7-e7cf3cd640ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:54.060944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57d805c3-c697-4161-b6d7-e7cf3cd640ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:54.061395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57d805c3-c697-4161-b6d7-e7cf3cd640ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:54.061820 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57d805c3-c697-4161-b6d7-e7cf3cd640ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:54.090389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-57d805c3-c697-4161-b6d7-e7cf3cd640ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045695 Sep 08 13:59:54.090947 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 499/887] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:54 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:54.422685 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c9fd720e-2795-4ddb-a4c3-5f3ac1c22681 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:54.423294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c9fd720e-2795-4ddb-a4c3-5f3ac1c22681 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:54.430196 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c9fd720e-2795-4ddb-a4c3-5f3ac1c22681 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:54.430684 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c9fd720e-2795-4ddb-a4c3-5f3ac1c22681 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:54.431116 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c9fd720e-2795-4ddb-a4c3-5f3ac1c22681 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:54.431839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c9fd720e-2795-4ddb-a4c3-5f3ac1c22681 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:54.432286 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c9fd720e-2795-4ddb-a4c3-5f3ac1c22681 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:54.432705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c9fd720e-2795-4ddb-a4c3-5f3ac1c22681 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:54.582768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c9fd720e-2795-4ddb-a4c3-5f3ac1c22681 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:54.591807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c9fd720e-2795-4ddb-a4c3-5f3ac1c22681 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1754 microversion: 2.1 time: 0.172905 Sep 08 13:59:54.592773 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 389/888] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:54 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1754 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:54.648824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-307c7a39-65ee-4a29-aab7-edcfc840fecd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:54.649589 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-307c7a39-65ee-4a29-aab7-edcfc840fecd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:54.703724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-307c7a39-65ee-4a29-aab7-edcfc840fecd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:54.710597 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-307c7a39-65ee-4a29-aab7-edcfc840fecd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 200 len: 1240 microversion: 2.1 time: 0.065658 Sep 08 13:59:54.711152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 500/889] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:54 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 1240 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:55.103387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0cac7ba5-4659-4e51-af6c-ae8eee35854c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:55.105585 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0cac7ba5-4659-4e51-af6c-ae8eee35854c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:55.111629 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0cac7ba5-4659-4e51-af6c-ae8eee35854c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:55.112103 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0cac7ba5-4659-4e51-af6c-ae8eee35854c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:55.112527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0cac7ba5-4659-4e51-af6c-ae8eee35854c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:55.113209 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0cac7ba5-4659-4e51-af6c-ae8eee35854c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:55.113651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0cac7ba5-4659-4e51-af6c-ae8eee35854c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:55.114062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0cac7ba5-4659-4e51-af6c-ae8eee35854c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:55.144137 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0cac7ba5-4659-4e51-af6c-ae8eee35854c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044745 Sep 08 13:59:55.144687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 390/890] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:55 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:59:55.171464 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-53e568df-0f45-43cd-8055-8083f64ce276 req-25edd0a1-bbcf-4cd5-a579-bebf829862aa service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "19db4aa2-f574-43fa-bbf8-c3042b243152", "tag": "2ebf3a50-af15-4ecc-820b-87ad0a9ff652"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 13:59:55.180943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-53e568df-0f45-43cd-8055-8083f64ce276 req-25edd0a1-bbcf-4cd5-a579-bebf829862aa service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:55.181926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-53e568df-0f45-43cd-8055-8083f64ce276 req-25edd0a1-bbcf-4cd5-a579-bebf829862aa service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:55.182446 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-53e568df-0f45-43cd-8055-8083f64ce276 req-25edd0a1-bbcf-4cd5-a579-bebf829862aa service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:55.194129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-53e568df-0f45-43cd-8055-8083f64ce276 req-25edd0a1-bbcf-4cd5-a579-bebf829862aa service nova] Creating event network-changed:2ebf3a50-af15-4ecc-820b-87ad0a9ff652 for instance 19db4aa2-f574-43fa-bbf8-c3042b243152 on hv-cinder-85646 Sep 08 13:59:55.199746 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-53e568df-0f45-43cd-8055-8083f64ce276 req-25edd0a1-bbcf-4cd5-a579-bebf829862aa service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032107 Sep 08 13:59:55.200347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 501/891] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 13:59:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 13:59:55.615856 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bc67db6b-57ff-462c-955b-01a1c7e159ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:55.616640 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bc67db6b-57ff-462c-955b-01a1c7e159ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:55.623775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc67db6b-57ff-462c-955b-01a1c7e159ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:55.624265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc67db6b-57ff-462c-955b-01a1c7e159ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:55.624890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc67db6b-57ff-462c-955b-01a1c7e159ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:55.625618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc67db6b-57ff-462c-955b-01a1c7e159ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:55.626074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc67db6b-57ff-462c-955b-01a1c7e159ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:55.626534 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc67db6b-57ff-462c-955b-01a1c7e159ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:55.735905 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2d9729e4-149e-439f-b255-a72c24f1a1ba tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:55.736602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2d9729e4-149e-439f-b255-a72c24f1a1ba tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:55.816038 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2d9729e4-149e-439f-b255-a72c24f1a1ba tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:55.827598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2d9729e4-149e-439f-b255-a72c24f1a1ba tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 200 len: 1240 microversion: 2.1 time: 0.095941 Sep 08 13:59:55.829825 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 502/892] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:55 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 1240 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 13:59:55.895106 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-bc67db6b-57ff-462c-955b-01a1c7e159ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:55.904865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bc67db6b-57ff-462c-955b-01a1c7e159ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1754 microversion: 2.1 time: 0.292860 Sep 08 13:59:55.905450 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 391/893] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:55 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1754 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:56.158540 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-791be21c-2f23-4d5e-a66a-fdf7285b68fa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:56.160815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-791be21c-2f23-4d5e-a66a-fdf7285b68fa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:56.167787 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-791be21c-2f23-4d5e-a66a-fdf7285b68fa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:56.168272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-791be21c-2f23-4d5e-a66a-fdf7285b68fa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:56.168706 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-791be21c-2f23-4d5e-a66a-fdf7285b68fa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:56.169402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-791be21c-2f23-4d5e-a66a-fdf7285b68fa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:56.169862 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-791be21c-2f23-4d5e-a66a-fdf7285b68fa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:56.170327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-791be21c-2f23-4d5e-a66a-fdf7285b68fa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:56.212941 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-791be21c-2f23-4d5e-a66a-fdf7285b68fa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058992 Sep 08 13:59:56.213575 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 503/894] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:56 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:59:56.853097 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-36c2d94c-d620-4e69-922c-3f5fdb57f3a5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:56.853727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-36c2d94c-d620-4e69-922c-3f5fdb57f3a5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:56.915864 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-36c2d94c-d620-4e69-922c-3f5fdb57f3a5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:56.922798 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-36c2d94c-d620-4e69-922c-3f5fdb57f3a5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 200 len: 1240 microversion: 2.1 time: 0.073609 Sep 08 13:59:56.923366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 392/895] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:56 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 1240 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:56.929330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ab0a0fe2-e579-4443-beab-d128140b935a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:56.929949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ab0a0fe2-e579-4443-beab-d128140b935a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:56.936615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab0a0fe2-e579-4443-beab-d128140b935a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:56.937104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab0a0fe2-e579-4443-beab-d128140b935a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:56.937547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab0a0fe2-e579-4443-beab-d128140b935a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:56.938267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab0a0fe2-e579-4443-beab-d128140b935a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:56.938724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab0a0fe2-e579-4443-beab-d128140b935a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:56.939152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab0a0fe2-e579-4443-beab-d128140b935a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:57.079903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ab0a0fe2-e579-4443-beab-d128140b935a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:57.088292 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ab0a0fe2-e579-4443-beab-d128140b935a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1754 microversion: 2.1 time: 0.162684 Sep 08 13:59:57.088960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 504/896] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:56 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1754 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:57.226602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-01ea6bae-f8c9-4ba5-9dd1-8f96e6eaea79 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:57.229266 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-01ea6bae-f8c9-4ba5-9dd1-8f96e6eaea79 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:57.236194 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-01ea6bae-f8c9-4ba5-9dd1-8f96e6eaea79 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:57.236687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-01ea6bae-f8c9-4ba5-9dd1-8f96e6eaea79 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:57.237126 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-01ea6bae-f8c9-4ba5-9dd1-8f96e6eaea79 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:57.237823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-01ea6bae-f8c9-4ba5-9dd1-8f96e6eaea79 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:57.238325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-01ea6bae-f8c9-4ba5-9dd1-8f96e6eaea79 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:57.238732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-01ea6bae-f8c9-4ba5-9dd1-8f96e6eaea79 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:57.268743 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-01ea6bae-f8c9-4ba5-9dd1-8f96e6eaea79 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046159 Sep 08 13:59:57.270656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 393/897] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:57 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:57.946602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-59d1ae70-5604-4d99-b877-2e02d02052b0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:57.947713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-59d1ae70-5604-4d99-b877-2e02d02052b0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:57.954115 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-59d1ae70-5604-4d99-b877-2e02d02052b0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:57.954949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-59d1ae70-5604-4d99-b877-2e02d02052b0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:57.955702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-59d1ae70-5604-4d99-b877-2e02d02052b0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:57.956917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-59d1ae70-5604-4d99-b877-2e02d02052b0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:57.957726 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-59d1ae70-5604-4d99-b877-2e02d02052b0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:57.958512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-59d1ae70-5604-4d99-b877-2e02d02052b0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:58.031787 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-59d1ae70-5604-4d99-b877-2e02d02052b0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:58.039184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-59d1ae70-5604-4d99-b877-2e02d02052b0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 200 len: 1244 microversion: 2.1 time: 0.096591 Sep 08 13:59:58.039764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 505/898] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:57 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 1244 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:58.111842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d093736b-8fc6-4035-84ee-e7cf64ca0937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:58.112451 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d093736b-8fc6-4035-84ee-e7cf64ca0937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:58.119796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d093736b-8fc6-4035-84ee-e7cf64ca0937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:58.120276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d093736b-8fc6-4035-84ee-e7cf64ca0937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:58.120711 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d093736b-8fc6-4035-84ee-e7cf64ca0937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:58.121407 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d093736b-8fc6-4035-84ee-e7cf64ca0937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:58.121863 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d093736b-8fc6-4035-84ee-e7cf64ca0937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:58.122322 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d093736b-8fc6-4035-84ee-e7cf64ca0937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:58.265009 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d093736b-8fc6-4035-84ee-e7cf64ca0937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:58.273282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d093736b-8fc6-4035-84ee-e7cf64ca0937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1754 microversion: 2.1 time: 0.165298 Sep 08 13:59:58.273849 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 394/899] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:58 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1754 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:58.281275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-25f1699f-af80-4431-8eff-ee389c075b58 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:58.283454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-25f1699f-af80-4431-8eff-ee389c075b58 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:58.289452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25f1699f-af80-4431-8eff-ee389c075b58 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:58.289936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25f1699f-af80-4431-8eff-ee389c075b58 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:58.290412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25f1699f-af80-4431-8eff-ee389c075b58 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:58.291108 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25f1699f-af80-4431-8eff-ee389c075b58 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:58.291561 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25f1699f-af80-4431-8eff-ee389c075b58 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:58.291986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25f1699f-af80-4431-8eff-ee389c075b58 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:58.324122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-25f1699f-af80-4431-8eff-ee389c075b58 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046677 Sep 08 13:59:58.324685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 506/900] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:58 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 13:59:59.063392 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ca1e4935-295d-4dec-8455-3d196674b082 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:59.063998 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ca1e4935-295d-4dec-8455-3d196674b082 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:59.070107 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca1e4935-295d-4dec-8455-3d196674b082 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:59.070617 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca1e4935-295d-4dec-8455-3d196674b082 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:59.071054 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca1e4935-295d-4dec-8455-3d196674b082 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:59.071746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca1e4935-295d-4dec-8455-3d196674b082 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:59.072200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca1e4935-295d-4dec-8455-3d196674b082 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:59.072632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca1e4935-295d-4dec-8455-3d196674b082 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:59.136990 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ca1e4935-295d-4dec-8455-3d196674b082 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:59.144868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ca1e4935-295d-4dec-8455-3d196674b082 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 200 len: 1236 microversion: 2.1 time: 0.085419 Sep 08 13:59:59.145533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 395/901] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:59 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 1236 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 13:59:59.297272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f38884d0-d0d5-4e6d-828b-f56d98e7655b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:59.297855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f38884d0-d0d5-4e6d-828b-f56d98e7655b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:59.304479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f38884d0-d0d5-4e6d-828b-f56d98e7655b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:59.304972 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f38884d0-d0d5-4e6d-828b-f56d98e7655b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:59.305402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f38884d0-d0d5-4e6d-828b-f56d98e7655b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:59.306170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f38884d0-d0d5-4e6d-828b-f56d98e7655b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:59.306621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f38884d0-d0d5-4e6d-828b-f56d98e7655b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:59.307036 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f38884d0-d0d5-4e6d-828b-f56d98e7655b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:59.337357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2cd6eb22-fcb0-4c1b-8b00-5ec388b14ebc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 13:59:59.340377 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2cd6eb22-fcb0-4c1b-8b00-5ec388b14ebc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 13:59:59.348150 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2cd6eb22-fcb0-4c1b-8b00-5ec388b14ebc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:59.348778 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2cd6eb22-fcb0-4c1b-8b00-5ec388b14ebc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:59.349227 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2cd6eb22-fcb0-4c1b-8b00-5ec388b14ebc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:59.349940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2cd6eb22-fcb0-4c1b-8b00-5ec388b14ebc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 13:59:59.350612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2cd6eb22-fcb0-4c1b-8b00-5ec388b14ebc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 13:59:59.351191 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2cd6eb22-fcb0-4c1b-8b00-5ec388b14ebc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 13:59:59.388646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2cd6eb22-fcb0-4c1b-8b00-5ec388b14ebc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055607 Sep 08 13:59:59.391231 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 396/902] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 13:59:59 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 13:59:59.472301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f38884d0-d0d5-4e6d-828b-f56d98e7655b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 13:59:59.480532 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f38884d0-d0d5-4e6d-828b-f56d98e7655b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1754 microversion: 2.1 time: 0.187493 Sep 08 13:59:59.481092 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 507/903] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 13:59:59 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1754 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:00.170931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7852c487-3a2c-4ed6-822e-6a01846a43e5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:00.171573 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7852c487-3a2c-4ed6-822e-6a01846a43e5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:00.178057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7852c487-3a2c-4ed6-822e-6a01846a43e5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:00.178585 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7852c487-3a2c-4ed6-822e-6a01846a43e5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:00.179017 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7852c487-3a2c-4ed6-822e-6a01846a43e5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:00.179808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7852c487-3a2c-4ed6-822e-6a01846a43e5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:00.180260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7852c487-3a2c-4ed6-822e-6a01846a43e5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:00.180682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7852c487-3a2c-4ed6-822e-6a01846a43e5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:00.252937 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7852c487-3a2c-4ed6-822e-6a01846a43e5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:00.260679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7852c487-3a2c-4ed6-822e-6a01846a43e5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 200 len: 1292 microversion: 2.1 time: 0.094959 Sep 08 14:00:00.261253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 397/904] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:00 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 1292 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:00.405484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-dcba8c23-216f-4a70-a964-2a4e299acc4e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:00.408897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-dcba8c23-216f-4a70-a964-2a4e299acc4e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:00.431068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dcba8c23-216f-4a70-a964-2a4e299acc4e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:00.431565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dcba8c23-216f-4a70-a964-2a4e299acc4e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:00.432000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dcba8c23-216f-4a70-a964-2a4e299acc4e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:00.432703 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dcba8c23-216f-4a70-a964-2a4e299acc4e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:00.433175 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dcba8c23-216f-4a70-a964-2a4e299acc4e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:00.433574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dcba8c23-216f-4a70-a964-2a4e299acc4e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:00.465831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-dcba8c23-216f-4a70-a964-2a4e299acc4e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.068715 Sep 08 14:00:00.466447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 508/905] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:00 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:00.505050 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e39cd573-53a4-4e9d-9f48-694fe6240a45 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:00.505563 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e39cd573-53a4-4e9d-9f48-694fe6240a45 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:00.511706 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e39cd573-53a4-4e9d-9f48-694fe6240a45 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:00.512197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e39cd573-53a4-4e9d-9f48-694fe6240a45 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:00.512625 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e39cd573-53a4-4e9d-9f48-694fe6240a45 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:00.513346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e39cd573-53a4-4e9d-9f48-694fe6240a45 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:00.513809 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e39cd573-53a4-4e9d-9f48-694fe6240a45 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:00.514449 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e39cd573-53a4-4e9d-9f48-694fe6240a45 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:00.723643 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e39cd573-53a4-4e9d-9f48-694fe6240a45 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:00.732987 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e39cd573-53a4-4e9d-9f48-694fe6240a45 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1754 microversion: 2.1 time: 0.232019 Sep 08 14:00:00.733650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 398/906] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:00 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1754 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:01.284493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4b0a3028-8c35-4959-8cb4-c966f6b08541 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:01.285109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4b0a3028-8c35-4959-8cb4-c966f6b08541 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:01.292392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4b0a3028-8c35-4959-8cb4-c966f6b08541 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:01.292888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4b0a3028-8c35-4959-8cb4-c966f6b08541 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:01.293620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4b0a3028-8c35-4959-8cb4-c966f6b08541 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:01.294370 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4b0a3028-8c35-4959-8cb4-c966f6b08541 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:01.294879 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4b0a3028-8c35-4959-8cb4-c966f6b08541 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:01.299213 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4b0a3028-8c35-4959-8cb4-c966f6b08541 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:01.312180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:01.437596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4b0a3028-8c35-4959-8cb4-c966f6b08541 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:01.445893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4b0a3028-8c35-4959-8cb4-c966f6b08541 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 200 len: 1365 microversion: 2.1 time: 0.165621 Sep 08 14:00:01.446529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 509/907] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:01 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 1365 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:01.480236 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-32f2f869-b703-4dbd-bbe1-6d9d4c42cc03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:01.482439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-32f2f869-b703-4dbd-bbe1-6d9d4c42cc03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:01.489613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-32f2f869-b703-4dbd-bbe1-6d9d4c42cc03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:01.490105 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-32f2f869-b703-4dbd-bbe1-6d9d4c42cc03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:01.490823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-32f2f869-b703-4dbd-bbe1-6d9d4c42cc03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:01.491717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-32f2f869-b703-4dbd-bbe1-6d9d4c42cc03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:01.492181 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-32f2f869-b703-4dbd-bbe1-6d9d4c42cc03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:01.492611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-32f2f869-b703-4dbd-bbe1-6d9d4c42cc03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:01.525165 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-32f2f869-b703-4dbd-bbe1-6d9d4c42cc03 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049468 Sep 08 14:00:01.526371 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 399/908] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:01 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:01.757709 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-15171508-33a8-4b71-83ef-d4e219717a87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:01.758932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-15171508-33a8-4b71-83ef-d4e219717a87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:01.766526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-15171508-33a8-4b71-83ef-d4e219717a87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:01.767130 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-15171508-33a8-4b71-83ef-d4e219717a87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:01.767579 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-15171508-33a8-4b71-83ef-d4e219717a87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:01.768299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-15171508-33a8-4b71-83ef-d4e219717a87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:01.768766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-15171508-33a8-4b71-83ef-d4e219717a87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:01.769204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-15171508-33a8-4b71-83ef-d4e219717a87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:01.918045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-15171508-33a8-4b71-83ef-d4e219717a87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:01.926806 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-15171508-33a8-4b71-83ef-d4e219717a87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1754 microversion: 2.1 time: 0.173884 Sep 08 14:00:01.927756 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 510/909] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:01 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1754 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:02.302751 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-a6195589-6218-419a-a9ec-91ee9b6aa48b req-e25f57a8-c555-4617-8c6d-0867e97cc10e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5a9f0e69-51d0-4a26-89d2-91c47d197556", "tag": "afc328f7-6869-4252-8062-09c20950ee2a"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:00:02.313453 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-a6195589-6218-419a-a9ec-91ee9b6aa48b req-e25f57a8-c555-4617-8c6d-0867e97cc10e service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:02.313934 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-a6195589-6218-419a-a9ec-91ee9b6aa48b req-e25f57a8-c555-4617-8c6d-0867e97cc10e service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:02.314395 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-a6195589-6218-419a-a9ec-91ee9b6aa48b req-e25f57a8-c555-4617-8c6d-0867e97cc10e service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:02.326753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-a6195589-6218-419a-a9ec-91ee9b6aa48b req-e25f57a8-c555-4617-8c6d-0867e97cc10e service nova] Creating event network-changed:afc328f7-6869-4252-8062-09c20950ee2a for instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 on hv-cinder-85646 Sep 08 14:00:02.332202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-a6195589-6218-419a-a9ec-91ee9b6aa48b req-e25f57a8-c555-4617-8c6d-0867e97cc10e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032691 Sep 08 14:00:02.332607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 400/910] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:00:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:02.469296 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-92393621-90be-4982-a454-e6613bf241d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:02.469892 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-92393621-90be-4982-a454-e6613bf241d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:02.476540 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-92393621-90be-4982-a454-e6613bf241d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:02.477017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-92393621-90be-4982-a454-e6613bf241d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:02.477470 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-92393621-90be-4982-a454-e6613bf241d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:02.478168 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-92393621-90be-4982-a454-e6613bf241d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:02.479277 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-92393621-90be-4982-a454-e6613bf241d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:02.479740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-92393621-90be-4982-a454-e6613bf241d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:02.540774 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0ab573bb-948c-49c6-b004-b2471ab3ad77 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:02.543904 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0ab573bb-948c-49c6-b004-b2471ab3ad77 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:02.550948 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ab573bb-948c-49c6-b004-b2471ab3ad77 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:02.552882 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ab573bb-948c-49c6-b004-b2471ab3ad77 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:02.552882 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ab573bb-948c-49c6-b004-b2471ab3ad77 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:02.552882 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ab573bb-948c-49c6-b004-b2471ab3ad77 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:02.553315 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ab573bb-948c-49c6-b004-b2471ab3ad77 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:02.553709 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ab573bb-948c-49c6-b004-b2471ab3ad77 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:02.589863 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0ab573bb-948c-49c6-b004-b2471ab3ad77 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054550 Sep 08 14:00:02.591209 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 401/911] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:02 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:00:02.628402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-92393621-90be-4982-a454-e6613bf241d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:02.636584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-92393621-90be-4982-a454-e6613bf241d2 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 200 len: 1365 microversion: 2.1 time: 0.171047 Sep 08 14:00:02.637558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 511/912] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:02 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 1365 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:02.952681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-69dca045-f8d4-4d6d-a8ed-23583d8ba28a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:02.953283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-69dca045-f8d4-4d6d-a8ed-23583d8ba28a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:02.960792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-69dca045-f8d4-4d6d-a8ed-23583d8ba28a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:02.961339 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-69dca045-f8d4-4d6d-a8ed-23583d8ba28a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:02.961757 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-69dca045-f8d4-4d6d-a8ed-23583d8ba28a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:02.962498 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-69dca045-f8d4-4d6d-a8ed-23583d8ba28a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:02.963584 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-69dca045-f8d4-4d6d-a8ed-23583d8ba28a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:02.964005 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-69dca045-f8d4-4d6d-a8ed-23583d8ba28a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:03.208938 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-69dca045-f8d4-4d6d-a8ed-23583d8ba28a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:03.217624 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-69dca045-f8d4-4d6d-a8ed-23583d8ba28a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1754 microversion: 2.1 time: 0.270335 Sep 08 14:00:03.218260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 402/913] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:02 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1754 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:03.599322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=2 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:03.604451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9465676e-0c34-43a5-931b-3dd626b6901e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:03.606715 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9465676e-0c34-43a5-931b-3dd626b6901e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:03.612964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9465676e-0c34-43a5-931b-3dd626b6901e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:03.613448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9465676e-0c34-43a5-931b-3dd626b6901e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:03.616404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9465676e-0c34-43a5-931b-3dd626b6901e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:03.616404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9465676e-0c34-43a5-931b-3dd626b6901e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:03.616404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9465676e-0c34-43a5-931b-3dd626b6901e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:03.616404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9465676e-0c34-43a5-931b-3dd626b6901e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:03.646378 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9465676e-0c34-43a5-931b-3dd626b6901e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047877 Sep 08 14:00:03.647217 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 512/914] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:03 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:03.661977 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5fcbe35a-c337-4a03-acfc-f0b28e23175f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:03.662616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5fcbe35a-c337-4a03-acfc-f0b28e23175f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:03.669565 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5fcbe35a-c337-4a03-acfc-f0b28e23175f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:03.670039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5fcbe35a-c337-4a03-acfc-f0b28e23175f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:03.670503 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5fcbe35a-c337-4a03-acfc-f0b28e23175f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:03.671192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5fcbe35a-c337-4a03-acfc-f0b28e23175f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:03.671656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5fcbe35a-c337-4a03-acfc-f0b28e23175f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:03.672050 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5fcbe35a-c337-4a03-acfc-f0b28e23175f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:03.805695 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5fcbe35a-c337-4a03-acfc-f0b28e23175f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:03.812194 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5fcbe35a-c337-4a03-acfc-f0b28e23175f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 200 len: 1365 microversion: 2.1 time: 0.153896 Sep 08 14:00:03.814106 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 403/915] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:03 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 1365 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:04.240915 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-944a832b-b697-405c-9647-82fe28a50792 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:04.241525 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-944a832b-b697-405c-9647-82fe28a50792 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:04.247871 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-944a832b-b697-405c-9647-82fe28a50792 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:04.248362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-944a832b-b697-405c-9647-82fe28a50792 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:04.248805 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-944a832b-b697-405c-9647-82fe28a50792 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:04.249505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-944a832b-b697-405c-9647-82fe28a50792 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:04.249965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-944a832b-b697-405c-9647-82fe28a50792 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:04.250432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-944a832b-b697-405c-9647-82fe28a50792 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:04.394045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-944a832b-b697-405c-9647-82fe28a50792 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:04.403057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-944a832b-b697-405c-9647-82fe28a50792 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1754 microversion: 2.1 time: 0.166008 Sep 08 14:00:04.403628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 513/916] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:04 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1754 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:04.659815 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-407bad92-f72a-4f23-8e37-ae1837fbfdc3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:04.662029 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-407bad92-f72a-4f23-8e37-ae1837fbfdc3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:04.668062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-407bad92-f72a-4f23-8e37-ae1837fbfdc3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:04.668533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-407bad92-f72a-4f23-8e37-ae1837fbfdc3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:04.668960 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-407bad92-f72a-4f23-8e37-ae1837fbfdc3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:04.669634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-407bad92-f72a-4f23-8e37-ae1837fbfdc3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:04.670078 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-407bad92-f72a-4f23-8e37-ae1837fbfdc3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:04.670524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-407bad92-f72a-4f23-8e37-ae1837fbfdc3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:04.690109 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:04.704776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-407bad92-f72a-4f23-8e37-ae1837fbfdc3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049302 Sep 08 14:00:04.705412 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 404/917] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:04 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:04.835141 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-649300fb-de58-40bb-bfef-42d786204f48 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:04.835744 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-649300fb-de58-40bb-bfef-42d786204f48 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:04.842804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-649300fb-de58-40bb-bfef-42d786204f48 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:04.843295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-649300fb-de58-40bb-bfef-42d786204f48 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:04.843737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-649300fb-de58-40bb-bfef-42d786204f48 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:04.844432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-649300fb-de58-40bb-bfef-42d786204f48 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:04.844888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-649300fb-de58-40bb-bfef-42d786204f48 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:04.845342 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-649300fb-de58-40bb-bfef-42d786204f48 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:04.983634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-649300fb-de58-40bb-bfef-42d786204f48 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:04.990642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-649300fb-de58-40bb-bfef-42d786204f48 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 200 len: 1365 microversion: 2.1 time: 0.159286 Sep 08 14:00:04.991218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 514/918] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:04 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 1365 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:05.426666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ccae104d-5d20-4045-835a-d676f238eb1d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:05.427296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ccae104d-5d20-4045-835a-d676f238eb1d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:05.440881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccae104d-5d20-4045-835a-d676f238eb1d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:05.440881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccae104d-5d20-4045-835a-d676f238eb1d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:05.440881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccae104d-5d20-4045-835a-d676f238eb1d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:05.440881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccae104d-5d20-4045-835a-d676f238eb1d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:05.441354 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccae104d-5d20-4045-835a-d676f238eb1d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:05.441354 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccae104d-5d20-4045-835a-d676f238eb1d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:05.589988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ccae104d-5d20-4045-835a-d676f238eb1d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:05.598542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ccae104d-5d20-4045-835a-d676f238eb1d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1754 microversion: 2.1 time: 0.175633 Sep 08 14:00:05.599130 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 405/919] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:05 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1754 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:05.718449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-20ba712c-3fa2-478c-b8fc-955d1bab7cc2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:05.720661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-20ba712c-3fa2-478c-b8fc-955d1bab7cc2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:05.726936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-20ba712c-3fa2-478c-b8fc-955d1bab7cc2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:05.727427 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-20ba712c-3fa2-478c-b8fc-955d1bab7cc2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:05.727892 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-20ba712c-3fa2-478c-b8fc-955d1bab7cc2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:05.729165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-20ba712c-3fa2-478c-b8fc-955d1bab7cc2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:05.729623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-20ba712c-3fa2-478c-b8fc-955d1bab7cc2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:05.730052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-20ba712c-3fa2-478c-b8fc-955d1bab7cc2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:05.760588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-20ba712c-3fa2-478c-b8fc-955d1bab7cc2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046299 Sep 08 14:00:05.761159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 515/920] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:05 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:06.014354 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9a8d7f8a-b384-4fbb-8e94-8d7689e16750 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:06.014938 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9a8d7f8a-b384-4fbb-8e94-8d7689e16750 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:06.021398 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9a8d7f8a-b384-4fbb-8e94-8d7689e16750 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:06.021885 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9a8d7f8a-b384-4fbb-8e94-8d7689e16750 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:06.022356 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9a8d7f8a-b384-4fbb-8e94-8d7689e16750 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:06.023316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9a8d7f8a-b384-4fbb-8e94-8d7689e16750 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:06.023755 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9a8d7f8a-b384-4fbb-8e94-8d7689e16750 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:06.024173 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9a8d7f8a-b384-4fbb-8e94-8d7689e16750 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:06.163866 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9a8d7f8a-b384-4fbb-8e94-8d7689e16750 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:06.173740 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9a8d7f8a-b384-4fbb-8e94-8d7689e16750 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 200 len: 1365 microversion: 2.1 time: 0.163332 Sep 08 14:00:06.174375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 406/921] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:06 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 1365 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:06.621697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1afba54b-9898-4155-ae1a-0419c6da92fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:06.622288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1afba54b-9898-4155-ae1a-0419c6da92fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:06.624104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=10 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:06.629297 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1afba54b-9898-4155-ae1a-0419c6da92fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:06.629924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1afba54b-9898-4155-ae1a-0419c6da92fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:06.630244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1afba54b-9898-4155-ae1a-0419c6da92fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:06.630940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1afba54b-9898-4155-ae1a-0419c6da92fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:06.631390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1afba54b-9898-4155-ae1a-0419c6da92fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:06.631844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1afba54b-9898-4155-ae1a-0419c6da92fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:06.774419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bf7eb200-a3f3-410f-b607-fda446ad6b85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:06.776753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bf7eb200-a3f3-410f-b607-fda446ad6b85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:06.779072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1afba54b-9898-4155-ae1a-0419c6da92fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:06.783246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf7eb200-a3f3-410f-b607-fda446ad6b85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:06.783732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf7eb200-a3f3-410f-b607-fda446ad6b85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:06.784164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf7eb200-a3f3-410f-b607-fda446ad6b85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:06.784858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf7eb200-a3f3-410f-b607-fda446ad6b85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:06.785310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf7eb200-a3f3-410f-b607-fda446ad6b85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:06.785729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf7eb200-a3f3-410f-b607-fda446ad6b85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:06.787785 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1afba54b-9898-4155-ae1a-0419c6da92fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1754 microversion: 2.1 time: 0.170607 Sep 08 14:00:06.788360 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 516/922] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:06 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1754 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:06.818573 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bf7eb200-a3f3-410f-b607-fda446ad6b85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048428 Sep 08 14:00:06.819138 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 407/923] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:06 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:00:07.198260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bafc7ea0-1258-4645-a632-a0d5147ff851 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:07.199765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bafc7ea0-1258-4645-a632-a0d5147ff851 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:07.205787 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bafc7ea0-1258-4645-a632-a0d5147ff851 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:07.206333 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bafc7ea0-1258-4645-a632-a0d5147ff851 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:07.206776 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bafc7ea0-1258-4645-a632-a0d5147ff851 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:07.207756 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bafc7ea0-1258-4645-a632-a0d5147ff851 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:07.207942 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bafc7ea0-1258-4645-a632-a0d5147ff851 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:07.208369 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bafc7ea0-1258-4645-a632-a0d5147ff851 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:07.496625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bafc7ea0-1258-4645-a632-a0d5147ff851 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:07.504859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bafc7ea0-1258-4645-a632-a0d5147ff851 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 200 len: 1365 microversion: 2.1 time: 0.311517 Sep 08 14:00:07.509535 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 517/924] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:07 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 1365 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:07.811833 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-92d40606-aaad-4b00-a5f7-0efac3e893ca tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:07.814406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-92d40606-aaad-4b00-a5f7-0efac3e893ca tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:07.814406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=8 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:07.819471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-92d40606-aaad-4b00-a5f7-0efac3e893ca tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:07.819953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-92d40606-aaad-4b00-a5f7-0efac3e893ca tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:07.820383 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-92d40606-aaad-4b00-a5f7-0efac3e893ca tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:07.821068 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-92d40606-aaad-4b00-a5f7-0efac3e893ca tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:07.822115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-92d40606-aaad-4b00-a5f7-0efac3e893ca tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:07.822565 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-92d40606-aaad-4b00-a5f7-0efac3e893ca tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:07.831088 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a1f97491-1ad3-480f-837a-95c1aae3d370 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:07.833475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a1f97491-1ad3-480f-837a-95c1aae3d370 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:07.839857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1f97491-1ad3-480f-837a-95c1aae3d370 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:07.843047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1f97491-1ad3-480f-837a-95c1aae3d370 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:07.843047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1f97491-1ad3-480f-837a-95c1aae3d370 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:07.843047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1f97491-1ad3-480f-837a-95c1aae3d370 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:07.843047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1f97491-1ad3-480f-837a-95c1aae3d370 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:07.843401 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1f97491-1ad3-480f-837a-95c1aae3d370 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:07.873309 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a1f97491-1ad3-480f-837a-95c1aae3d370 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046052 Sep 08 14:00:07.873868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 518/925] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:07 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:00:07.967133 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-92d40606-aaad-4b00-a5f7-0efac3e893ca tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:07.975289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-92d40606-aaad-4b00-a5f7-0efac3e893ca tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1754 microversion: 2.1 time: 0.167412 Sep 08 14:00:07.975849 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 408/926] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:07 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1754 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:08.529938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-73aa789d-2f4d-4c67-aa02-f7d7cbaa8ab8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:08.530576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-73aa789d-2f4d-4c67-aa02-f7d7cbaa8ab8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:08.536806 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73aa789d-2f4d-4c67-aa02-f7d7cbaa8ab8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:08.537294 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73aa789d-2f4d-4c67-aa02-f7d7cbaa8ab8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:08.537735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73aa789d-2f4d-4c67-aa02-f7d7cbaa8ab8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:08.538461 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73aa789d-2f4d-4c67-aa02-f7d7cbaa8ab8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:08.538917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73aa789d-2f4d-4c67-aa02-f7d7cbaa8ab8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:08.539341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73aa789d-2f4d-4c67-aa02-f7d7cbaa8ab8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:08.626171 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-44315c56-b052-4939-900b-aab891648ca6 req-af859d87-4d53-4be9-a0a1-83ade2bf1d06 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5a9f0e69-51d0-4a26-89d2-91c47d197556", "name": "network-vif-plugged", "status": "completed", "tag": "afc328f7-6869-4252-8062-09c20950ee2a"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:00:08.639555 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-44315c56-b052-4939-900b-aab891648ca6 req-af859d87-4d53-4be9-a0a1-83ade2bf1d06 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:08.644834 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-44315c56-b052-4939-900b-aab891648ca6 req-af859d87-4d53-4be9-a0a1-83ade2bf1d06 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:08.644834 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-44315c56-b052-4939-900b-aab891648ca6 req-af859d87-4d53-4be9-a0a1-83ade2bf1d06 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:08.653506 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-44315c56-b052-4939-900b-aab891648ca6 req-af859d87-4d53-4be9-a0a1-83ade2bf1d06 service nova] Creating event network-vif-plugged:afc328f7-6869-4252-8062-09c20950ee2a for instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 on hv-cinder-85646 Sep 08 14:00:08.658487 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-44315c56-b052-4939-900b-aab891648ca6 req-af859d87-4d53-4be9-a0a1-83ade2bf1d06 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036150 Sep 08 14:00:08.659173 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 409/927] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:00:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:00:08.681631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-73aa789d-2f4d-4c67-aa02-f7d7cbaa8ab8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:08.689327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-73aa789d-2f4d-4c67-aa02-f7d7cbaa8ab8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 200 len: 1365 microversion: 2.1 time: 0.164166 Sep 08 14:00:08.689889 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 519/928] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:08 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 1365 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:08.887238 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c8f5e8e9-bd85-480f-a95d-7c3be9535236 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:08.889441 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c8f5e8e9-bd85-480f-a95d-7c3be9535236 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:08.895890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c8f5e8e9-bd85-480f-a95d-7c3be9535236 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:08.896377 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c8f5e8e9-bd85-480f-a95d-7c3be9535236 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:08.896812 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c8f5e8e9-bd85-480f-a95d-7c3be9535236 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:08.897770 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c8f5e8e9-bd85-480f-a95d-7c3be9535236 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:08.898328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c8f5e8e9-bd85-480f-a95d-7c3be9535236 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:08.898755 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c8f5e8e9-bd85-480f-a95d-7c3be9535236 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:08.930003 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c8f5e8e9-bd85-480f-a95d-7c3be9535236 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048109 Sep 08 14:00:08.930622 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 410/929] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:08 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:08.999964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4d464f86-1521-4928-be3f-1d7d195b251a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:09.001023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4d464f86-1521-4928-be3f-1d7d195b251a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:09.013046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4d464f86-1521-4928-be3f-1d7d195b251a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:09.013625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4d464f86-1521-4928-be3f-1d7d195b251a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:09.014101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4d464f86-1521-4928-be3f-1d7d195b251a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:09.014845 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4d464f86-1521-4928-be3f-1d7d195b251a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:09.015386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4d464f86-1521-4928-be3f-1d7d195b251a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:09.015870 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4d464f86-1521-4928-be3f-1d7d195b251a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:09.167919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4d464f86-1521-4928-be3f-1d7d195b251a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:09.176384 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4d464f86-1521-4928-be3f-1d7d195b251a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1754 microversion: 2.1 time: 0.181816 Sep 08 14:00:09.176996 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 520/930] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:08 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1754 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:09.712237 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-39ab016b-0a0f-4fab-be32-62b9513b0f72 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:09.712847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-39ab016b-0a0f-4fab-be32-62b9513b0f72 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:09.719045 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39ab016b-0a0f-4fab-be32-62b9513b0f72 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:09.719659 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39ab016b-0a0f-4fab-be32-62b9513b0f72 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:09.720103 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39ab016b-0a0f-4fab-be32-62b9513b0f72 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:09.720801 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39ab016b-0a0f-4fab-be32-62b9513b0f72 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:09.721252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39ab016b-0a0f-4fab-be32-62b9513b0f72 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:09.721679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39ab016b-0a0f-4fab-be32-62b9513b0f72 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:09.863414 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-39ab016b-0a0f-4fab-be32-62b9513b0f72 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:09.872034 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-39ab016b-0a0f-4fab-be32-62b9513b0f72 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 200 len: 1365 microversion: 2.1 time: 0.163591 Sep 08 14:00:09.872607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 411/931] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:09 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 1365 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:09.943124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-48d26e41-2b8b-474a-bab9-8396fbb0b470 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:09.945263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-48d26e41-2b8b-474a-bab9-8396fbb0b470 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:09.951406 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-48d26e41-2b8b-474a-bab9-8396fbb0b470 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:09.951891 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-48d26e41-2b8b-474a-bab9-8396fbb0b470 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:09.952361 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-48d26e41-2b8b-474a-bab9-8396fbb0b470 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:09.953058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-48d26e41-2b8b-474a-bab9-8396fbb0b470 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:09.953510 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-48d26e41-2b8b-474a-bab9-8396fbb0b470 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:09.953934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-48d26e41-2b8b-474a-bab9-8396fbb0b470 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:09.985012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-48d26e41-2b8b-474a-bab9-8396fbb0b470 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046134 Sep 08 14:00:09.985563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 521/932] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:09 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:10.200635 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-64d50f7c-544f-4636-bc4b-7b9fe0de8793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:10.201200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-64d50f7c-544f-4636-bc4b-7b9fe0de8793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:10.207852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64d50f7c-544f-4636-bc4b-7b9fe0de8793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:10.208341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64d50f7c-544f-4636-bc4b-7b9fe0de8793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:10.208780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64d50f7c-544f-4636-bc4b-7b9fe0de8793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:10.209478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64d50f7c-544f-4636-bc4b-7b9fe0de8793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:10.209935 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64d50f7c-544f-4636-bc4b-7b9fe0de8793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:10.210387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64d50f7c-544f-4636-bc4b-7b9fe0de8793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:10.379921 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-64d50f7c-544f-4636-bc4b-7b9fe0de8793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:10.388934 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-64d50f7c-544f-4636-bc4b-7b9fe0de8793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1754 microversion: 2.1 time: 0.192235 Sep 08 14:00:10.389497 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 412/933] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:10 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1754 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:10.894988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-10735e7d-eadb-487b-90bd-7408fdc61dee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:10.895507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-10735e7d-eadb-487b-90bd-7408fdc61dee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:10.905150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-10735e7d-eadb-487b-90bd-7408fdc61dee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:10.905644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-10735e7d-eadb-487b-90bd-7408fdc61dee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:10.906082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-10735e7d-eadb-487b-90bd-7408fdc61dee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:10.907256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-10735e7d-eadb-487b-90bd-7408fdc61dee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:10.907815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-10735e7d-eadb-487b-90bd-7408fdc61dee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:10.908138 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-10735e7d-eadb-487b-90bd-7408fdc61dee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:10.997250 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-42596b83-8517-477e-a0ef-da9e47509995 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:10.999906 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-42596b83-8517-477e-a0ef-da9e47509995 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:11.006262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-42596b83-8517-477e-a0ef-da9e47509995 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:11.008875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-42596b83-8517-477e-a0ef-da9e47509995 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:11.008875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-42596b83-8517-477e-a0ef-da9e47509995 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:11.008875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-42596b83-8517-477e-a0ef-da9e47509995 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:11.008875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-42596b83-8517-477e-a0ef-da9e47509995 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:11.009220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-42596b83-8517-477e-a0ef-da9e47509995 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:11.049841 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-42596b83-8517-477e-a0ef-da9e47509995 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.056756 Sep 08 14:00:11.050528 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 413/934] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:10 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:00:11.052443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-10735e7d-eadb-487b-90bd-7408fdc61dee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:11.060802 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-10735e7d-eadb-487b-90bd-7408fdc61dee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 200 len: 1365 microversion: 2.1 time: 0.169673 Sep 08 14:00:11.063422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 522/935] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:10 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 1365 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:11.412912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4c1a3912-79a7-4fbb-95ef-37285886292c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:11.413526 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4c1a3912-79a7-4fbb-95ef-37285886292c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:11.416494 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=98,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:11.421158 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4c1a3912-79a7-4fbb-95ef-37285886292c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:11.421641 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4c1a3912-79a7-4fbb-95ef-37285886292c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:11.422084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4c1a3912-79a7-4fbb-95ef-37285886292c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:11.422794 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4c1a3912-79a7-4fbb-95ef-37285886292c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:11.423245 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4c1a3912-79a7-4fbb-95ef-37285886292c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:11.423667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4c1a3912-79a7-4fbb-95ef-37285886292c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:11.579768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4c1a3912-79a7-4fbb-95ef-37285886292c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:11.588582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4c1a3912-79a7-4fbb-95ef-37285886292c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1754 microversion: 2.1 time: 0.179696 Sep 08 14:00:11.589155 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 414/936] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:11 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1754 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:12.063042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2f8d58b3-0d18-4fe3-bd57-c536ede9842b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:12.065252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2f8d58b3-0d18-4fe3-bd57-c536ede9842b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:12.073285 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f8d58b3-0d18-4fe3-bd57-c536ede9842b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:12.073765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f8d58b3-0d18-4fe3-bd57-c536ede9842b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:12.074225 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f8d58b3-0d18-4fe3-bd57-c536ede9842b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:12.074919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f8d58b3-0d18-4fe3-bd57-c536ede9842b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:12.075364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f8d58b3-0d18-4fe3-bd57-c536ede9842b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:12.075783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f8d58b3-0d18-4fe3-bd57-c536ede9842b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:12.084810 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-42fdfc9a-8a3f-4221-960a-62bf666056fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:12.085422 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-42fdfc9a-8a3f-4221-960a-62bf666056fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:12.092023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-42fdfc9a-8a3f-4221-960a-62bf666056fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:12.092509 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-42fdfc9a-8a3f-4221-960a-62bf666056fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:12.092945 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-42fdfc9a-8a3f-4221-960a-62bf666056fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:12.093634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-42fdfc9a-8a3f-4221-960a-62bf666056fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:12.094090 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-42fdfc9a-8a3f-4221-960a-62bf666056fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:12.094548 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-42fdfc9a-8a3f-4221-960a-62bf666056fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:12.107745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2f8d58b3-0d18-4fe3-bd57-c536ede9842b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048797 Sep 08 14:00:12.112462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 523/937] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:12 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:12.228431 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-42fdfc9a-8a3f-4221-960a-62bf666056fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:12.236331 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-42fdfc9a-8a3f-4221-960a-62bf666056fd tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 200 len: 1554 microversion: 2.1 time: 0.155505 Sep 08 14:00:12.236904 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 415/938] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:12 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 1554 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:00:12.264597 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-13ac1f3f-5eab-4d79-81ce-624ef83f6943 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:12.265082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-13ac1f3f-5eab-4d79-81ce-624ef83f6943 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:12.270934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-13ac1f3f-5eab-4d79-81ce-624ef83f6943 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:12.271409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-13ac1f3f-5eab-4d79-81ce-624ef83f6943 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:12.272109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-13ac1f3f-5eab-4d79-81ce-624ef83f6943 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:12.272789 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-13ac1f3f-5eab-4d79-81ce-624ef83f6943 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:12.273239 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-13ac1f3f-5eab-4d79-81ce-624ef83f6943 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:12.273677 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-13ac1f3f-5eab-4d79-81ce-624ef83f6943 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:12.350502 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-13ac1f3f-5eab-4d79-81ce-624ef83f6943 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.089690 Sep 08 14:00:12.351109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 524/939] 10.222.0.25 () {58 vars in 1275 bytes} [Thu Sep 8 14:00:12 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556/os-interface => generated 284 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:00:12.613119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ffd15708-c39a-4cc6-b431-ea94dca38693 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:12.613845 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ffd15708-c39a-4cc6-b431-ea94dca38693 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:12.622277 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ffd15708-c39a-4cc6-b431-ea94dca38693 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:12.622768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ffd15708-c39a-4cc6-b431-ea94dca38693 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:12.623203 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ffd15708-c39a-4cc6-b431-ea94dca38693 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:12.623902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ffd15708-c39a-4cc6-b431-ea94dca38693 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:12.624353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ffd15708-c39a-4cc6-b431-ea94dca38693 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:12.624769 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ffd15708-c39a-4cc6-b431-ea94dca38693 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:12.774683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ffd15708-c39a-4cc6-b431-ea94dca38693 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:12.783042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ffd15708-c39a-4cc6-b431-ea94dca38693 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1754 microversion: 2.1 time: 0.173901 Sep 08 14:00:12.783622 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 416/940] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:12 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1754 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:13.120971 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-43118b40-6253-4bf8-af25-670032cf1688 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:13.123134 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-43118b40-6253-4bf8-af25-670032cf1688 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:13.129236 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43118b40-6253-4bf8-af25-670032cf1688 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:13.129721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43118b40-6253-4bf8-af25-670032cf1688 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:13.130832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43118b40-6253-4bf8-af25-670032cf1688 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:13.131568 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43118b40-6253-4bf8-af25-670032cf1688 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:13.132032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43118b40-6253-4bf8-af25-670032cf1688 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:13.132465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43118b40-6253-4bf8-af25-670032cf1688 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:13.163405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-43118b40-6253-4bf8-af25-670032cf1688 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046864 Sep 08 14:00:13.163884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 525/941] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:13 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:13.362692 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-3c2526a3-889c-4663-a2d3-5acdaad3426f req-6efcddfe-3ef8-4c77-86d4-0d93b607470e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5a9f0e69-51d0-4a26-89d2-91c47d197556", "tag": "afc328f7-6869-4252-8062-09c20950ee2a"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:00:13.372811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-3c2526a3-889c-4663-a2d3-5acdaad3426f req-6efcddfe-3ef8-4c77-86d4-0d93b607470e service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:13.373401 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-3c2526a3-889c-4663-a2d3-5acdaad3426f req-6efcddfe-3ef8-4c77-86d4-0d93b607470e service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:13.373703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-3c2526a3-889c-4663-a2d3-5acdaad3426f req-6efcddfe-3ef8-4c77-86d4-0d93b607470e service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:13.386481 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-3c2526a3-889c-4663-a2d3-5acdaad3426f req-6efcddfe-3ef8-4c77-86d4-0d93b607470e service nova] Creating event network-changed:afc328f7-6869-4252-8062-09c20950ee2a for instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 on hv-cinder-85646 Sep 08 14:00:13.392437 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-3c2526a3-889c-4663-a2d3-5acdaad3426f req-6efcddfe-3ef8-4c77-86d4-0d93b607470e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033663 Sep 08 14:00:13.395883 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 417/942] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:00:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:13.807214 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-69df2592-1ff3-4f03-9dee-7e579d0c0b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:13.807848 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-69df2592-1ff3-4f03-9dee-7e579d0c0b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:13.813894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69df2592-1ff3-4f03-9dee-7e579d0c0b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:13.814418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69df2592-1ff3-4f03-9dee-7e579d0c0b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:13.814865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69df2592-1ff3-4f03-9dee-7e579d0c0b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:13.815564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69df2592-1ff3-4f03-9dee-7e579d0c0b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:13.816051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69df2592-1ff3-4f03-9dee-7e579d0c0b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:13.816842 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69df2592-1ff3-4f03-9dee-7e579d0c0b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:13.832226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:13.967119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-69df2592-1ff3-4f03-9dee-7e579d0c0b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:00:13.982263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-69df2592-1ff3-4f03-9dee-7e579d0c0b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1757 microversion: 2.1 time: 0.179174 Sep 08 14:00:13.983306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 526/943] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:00:13 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1757 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:00:14.177374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a1cd8b5b-9f13-48d0-8a31-7cedac2d4e5a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:14.179666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a1cd8b5b-9f13-48d0-8a31-7cedac2d4e5a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:14.187360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1cd8b5b-9f13-48d0-8a31-7cedac2d4e5a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:14.187846 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1cd8b5b-9f13-48d0-8a31-7cedac2d4e5a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:14.188397 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1cd8b5b-9f13-48d0-8a31-7cedac2d4e5a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:14.189108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1cd8b5b-9f13-48d0-8a31-7cedac2d4e5a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:14.189560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1cd8b5b-9f13-48d0-8a31-7cedac2d4e5a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:14.189987 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1cd8b5b-9f13-48d0-8a31-7cedac2d4e5a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:14.221404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a1cd8b5b-9f13-48d0-8a31-7cedac2d4e5a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048334 Sep 08 14:00:14.221904 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 418/944] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:14 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:15.233190 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1f6fe4d5-5912-436e-8dc1-a8a5438ca281 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:15.235856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1f6fe4d5-5912-436e-8dc1-a8a5438ca281 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:15.243154 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1f6fe4d5-5912-436e-8dc1-a8a5438ca281 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:15.243657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1f6fe4d5-5912-436e-8dc1-a8a5438ca281 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:15.244721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1f6fe4d5-5912-436e-8dc1-a8a5438ca281 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:15.245458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1f6fe4d5-5912-436e-8dc1-a8a5438ca281 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:15.245966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1f6fe4d5-5912-436e-8dc1-a8a5438ca281 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:15.246465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1f6fe4d5-5912-436e-8dc1-a8a5438ca281 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:15.277831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1f6fe4d5-5912-436e-8dc1-a8a5438ca281 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048663 Sep 08 14:00:15.278419 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 527/945] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:15 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:16.291184 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6192f21d-ef40-471f-b5fc-c3f291ead56b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:16.293352 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6192f21d-ef40-471f-b5fc-c3f291ead56b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:16.300004 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6192f21d-ef40-471f-b5fc-c3f291ead56b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:16.300483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6192f21d-ef40-471f-b5fc-c3f291ead56b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:16.300917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6192f21d-ef40-471f-b5fc-c3f291ead56b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:16.301605 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6192f21d-ef40-471f-b5fc-c3f291ead56b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:16.302053 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6192f21d-ef40-471f-b5fc-c3f291ead56b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:16.302503 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6192f21d-ef40-471f-b5fc-c3f291ead56b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:16.333709 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6192f21d-ef40-471f-b5fc-c3f291ead56b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047122 Sep 08 14:00:16.334301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 419/946] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:16 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:17.347137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7b5e3793-9dbb-4f82-bc8d-5e845aa385a9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:17.349333 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7b5e3793-9dbb-4f82-bc8d-5e845aa385a9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:17.355969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b5e3793-9dbb-4f82-bc8d-5e845aa385a9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:17.356454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b5e3793-9dbb-4f82-bc8d-5e845aa385a9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:17.356882 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b5e3793-9dbb-4f82-bc8d-5e845aa385a9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:17.357576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b5e3793-9dbb-4f82-bc8d-5e845aa385a9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:17.358027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b5e3793-9dbb-4f82-bc8d-5e845aa385a9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:17.358482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b5e3793-9dbb-4f82-bc8d-5e845aa385a9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:17.389589 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7b5e3793-9dbb-4f82-bc8d-5e845aa385a9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046392 Sep 08 14:00:17.390179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 528/947] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:17 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:18.402609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-57a374b2-4884-4051-b9b8-7b7cf0f7d71a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:18.404744 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-57a374b2-4884-4051-b9b8-7b7cf0f7d71a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:18.411385 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-57a374b2-4884-4051-b9b8-7b7cf0f7d71a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:18.411870 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-57a374b2-4884-4051-b9b8-7b7cf0f7d71a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:18.412305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-57a374b2-4884-4051-b9b8-7b7cf0f7d71a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:18.412995 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-57a374b2-4884-4051-b9b8-7b7cf0f7d71a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:18.414003 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-57a374b2-4884-4051-b9b8-7b7cf0f7d71a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:18.414454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-57a374b2-4884-4051-b9b8-7b7cf0f7d71a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:18.443706 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-57a374b2-4884-4051-b9b8-7b7cf0f7d71a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045596 Sep 08 14:00:18.444275 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 420/948] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:18 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:19.456748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-218ee542-9db6-4b27-aa2b-92947a4e45b5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:19.458912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-218ee542-9db6-4b27-aa2b-92947a4e45b5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:19.465611 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-218ee542-9db6-4b27-aa2b-92947a4e45b5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:19.466098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-218ee542-9db6-4b27-aa2b-92947a4e45b5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:19.466570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-218ee542-9db6-4b27-aa2b-92947a4e45b5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:19.467269 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-218ee542-9db6-4b27-aa2b-92947a4e45b5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:19.467723 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-218ee542-9db6-4b27-aa2b-92947a4e45b5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:19.468152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-218ee542-9db6-4b27-aa2b-92947a4e45b5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:19.497828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-218ee542-9db6-4b27-aa2b-92947a4e45b5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045297 Sep 08 14:00:19.498422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 529/949] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:19 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:20.511200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-406f855a-830a-4bca-9199-7fc714a22b73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:20.513289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-406f855a-830a-4bca-9199-7fc714a22b73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:20.519913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-406f855a-830a-4bca-9199-7fc714a22b73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:20.523055 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-406f855a-830a-4bca-9199-7fc714a22b73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:20.523055 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-406f855a-830a-4bca-9199-7fc714a22b73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:20.523055 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-406f855a-830a-4bca-9199-7fc714a22b73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:20.523055 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-406f855a-830a-4bca-9199-7fc714a22b73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:20.523488 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-406f855a-830a-4bca-9199-7fc714a22b73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:20.551136 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-406f855a-830a-4bca-9199-7fc714a22b73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044583 Sep 08 14:00:20.551679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 421/950] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:20 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:21.564263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-585fbd87-0678-4a65-b4c4-21e3cfbe6500 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:21.566325 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-585fbd87-0678-4a65-b4c4-21e3cfbe6500 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:21.568601 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=3 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:21.573780 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-585fbd87-0678-4a65-b4c4-21e3cfbe6500 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:21.574296 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-585fbd87-0678-4a65-b4c4-21e3cfbe6500 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:21.574734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-585fbd87-0678-4a65-b4c4-21e3cfbe6500 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:21.575415 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-585fbd87-0678-4a65-b4c4-21e3cfbe6500 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:21.575869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-585fbd87-0678-4a65-b4c4-21e3cfbe6500 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:21.576281 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-585fbd87-0678-4a65-b4c4-21e3cfbe6500 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:21.609665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-585fbd87-0678-4a65-b4c4-21e3cfbe6500 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048746 Sep 08 14:00:21.609793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 530/951] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:21 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:22.624763 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2ac45300-45e2-4438-a5c8-b45424611992 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:22.627115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2ac45300-45e2-4438-a5c8-b45424611992 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:22.637871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2ac45300-45e2-4438-a5c8-b45424611992 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:22.637871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2ac45300-45e2-4438-a5c8-b45424611992 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:22.637871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2ac45300-45e2-4438-a5c8-b45424611992 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:22.637871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2ac45300-45e2-4438-a5c8-b45424611992 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:22.644917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2ac45300-45e2-4438-a5c8-b45424611992 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:22.644917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2ac45300-45e2-4438-a5c8-b45424611992 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:22.655060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:22.668436 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2ac45300-45e2-4438-a5c8-b45424611992 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047655 Sep 08 14:00:22.668988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 422/952] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:22 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:23.683152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-983cc86a-481a-416c-85bc-47529d941e2e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:23.685324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-983cc86a-481a-416c-85bc-47529d941e2e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:23.691880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-983cc86a-481a-416c-85bc-47529d941e2e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:23.692371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-983cc86a-481a-416c-85bc-47529d941e2e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:23.692813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-983cc86a-481a-416c-85bc-47529d941e2e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:23.693512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-983cc86a-481a-416c-85bc-47529d941e2e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:23.693970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-983cc86a-481a-416c-85bc-47529d941e2e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:23.694438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-983cc86a-481a-416c-85bc-47529d941e2e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:23.723754 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-983cc86a-481a-416c-85bc-47529d941e2e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044944 Sep 08 14:00:23.724306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 531/953] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:23 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:24.737039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-adbd515a-d06d-4e22-8f2d-7c7bc31d0c77 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:24.739188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-adbd515a-d06d-4e22-8f2d-7c7bc31d0c77 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:24.746700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-adbd515a-d06d-4e22-8f2d-7c7bc31d0c77 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:24.747184 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-adbd515a-d06d-4e22-8f2d-7c7bc31d0c77 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:24.747618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-adbd515a-d06d-4e22-8f2d-7c7bc31d0c77 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:24.748388 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-adbd515a-d06d-4e22-8f2d-7c7bc31d0c77 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:24.748789 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-adbd515a-d06d-4e22-8f2d-7c7bc31d0c77 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:24.749211 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-adbd515a-d06d-4e22-8f2d-7c7bc31d0c77 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:24.778758 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-adbd515a-d06d-4e22-8f2d-7c7bc31d0c77 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046449 Sep 08 14:00:24.779312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 423/954] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:24 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:25.791366 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4381c1c1-4f15-4f34-a7f1-bd879a501e52 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:25.793518 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4381c1c1-4f15-4f34-a7f1-bd879a501e52 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:25.799770 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4381c1c1-4f15-4f34-a7f1-bd879a501e52 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:25.800257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4381c1c1-4f15-4f34-a7f1-bd879a501e52 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:25.800692 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4381c1c1-4f15-4f34-a7f1-bd879a501e52 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:25.801382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4381c1c1-4f15-4f34-a7f1-bd879a501e52 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:25.801836 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4381c1c1-4f15-4f34-a7f1-bd879a501e52 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:25.802825 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4381c1c1-4f15-4f34-a7f1-bd879a501e52 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:25.833082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4381c1c1-4f15-4f34-a7f1-bd879a501e52 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045745 Sep 08 14:00:25.833703 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 532/955] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:25 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:26.845772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e15248b8-6515-41c0-b069-ed64fe67c014 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:26.847927 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e15248b8-6515-41c0-b069-ed64fe67c014 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:26.854850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e15248b8-6515-41c0-b069-ed64fe67c014 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:26.855338 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e15248b8-6515-41c0-b069-ed64fe67c014 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:26.855891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e15248b8-6515-41c0-b069-ed64fe67c014 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:26.856594 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e15248b8-6515-41c0-b069-ed64fe67c014 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:26.857049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e15248b8-6515-41c0-b069-ed64fe67c014 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:26.857475 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e15248b8-6515-41c0-b069-ed64fe67c014 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:26.886731 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e15248b8-6515-41c0-b069-ed64fe67c014 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045155 Sep 08 14:00:26.887278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 424/956] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:26 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:27.900353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7f67c19c-eb1d-4251-bfbc-a8dd4141c589 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:27.902503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7f67c19c-eb1d-4251-bfbc-a8dd4141c589 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:27.909300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f67c19c-eb1d-4251-bfbc-a8dd4141c589 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:27.909796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f67c19c-eb1d-4251-bfbc-a8dd4141c589 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:27.910276 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f67c19c-eb1d-4251-bfbc-a8dd4141c589 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:27.910983 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f67c19c-eb1d-4251-bfbc-a8dd4141c589 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:27.912061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f67c19c-eb1d-4251-bfbc-a8dd4141c589 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:27.912495 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f67c19c-eb1d-4251-bfbc-a8dd4141c589 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:27.941426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7f67c19c-eb1d-4251-bfbc-a8dd4141c589 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045746 Sep 08 14:00:27.941969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 533/957] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:27 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:28.953840 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f07b95a1-b63b-4903-80df-d256250eb610 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:28.955898 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f07b95a1-b63b-4903-80df-d256250eb610 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:28.962971 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f07b95a1-b63b-4903-80df-d256250eb610 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:28.963451 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f07b95a1-b63b-4903-80df-d256250eb610 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:28.963888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f07b95a1-b63b-4903-80df-d256250eb610 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:28.964650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f07b95a1-b63b-4903-80df-d256250eb610 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:28.965103 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f07b95a1-b63b-4903-80df-d256250eb610 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:28.965529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f07b95a1-b63b-4903-80df-d256250eb610 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:28.995563 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f07b95a1-b63b-4903-80df-d256250eb610 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045730 Sep 08 14:00:28.996124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 425/958] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:28 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:30.009021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-26383c25-01a2-4864-935c-1aa9e9860505 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:30.011139 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-26383c25-01a2-4864-935c-1aa9e9860505 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:30.017675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26383c25-01a2-4864-935c-1aa9e9860505 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:30.018189 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26383c25-01a2-4864-935c-1aa9e9860505 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:30.018895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26383c25-01a2-4864-935c-1aa9e9860505 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:30.019611 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26383c25-01a2-4864-935c-1aa9e9860505 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:30.020623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26383c25-01a2-4864-935c-1aa9e9860505 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:30.021144 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26383c25-01a2-4864-935c-1aa9e9860505 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:30.049384 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-26383c25-01a2-4864-935c-1aa9e9860505 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045008 Sep 08 14:00:30.049937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 534/959] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:30 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:31.061652 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-abf9b1df-c7f9-47d3-87d3-0b3ee0765bfc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:31.063850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-abf9b1df-c7f9-47d3-87d3-0b3ee0765bfc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:31.071198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abf9b1df-c7f9-47d3-87d3-0b3ee0765bfc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:31.072032 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abf9b1df-c7f9-47d3-87d3-0b3ee0765bfc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:31.072032 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abf9b1df-c7f9-47d3-87d3-0b3ee0765bfc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:31.075840 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abf9b1df-c7f9-47d3-87d3-0b3ee0765bfc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:31.075840 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abf9b1df-c7f9-47d3-87d3-0b3ee0765bfc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:31.075840 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abf9b1df-c7f9-47d3-87d3-0b3ee0765bfc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:31.104332 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-abf9b1df-c7f9-47d3-87d3-0b3ee0765bfc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046561 Sep 08 14:00:31.104829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 426/960] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:31 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:32.117013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-72a62c5d-7f0e-4bb3-ab68-0debe481fed0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:32.119240 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-72a62c5d-7f0e-4bb3-ab68-0debe481fed0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:32.125784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-72a62c5d-7f0e-4bb3-ab68-0debe481fed0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:32.126312 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-72a62c5d-7f0e-4bb3-ab68-0debe481fed0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:32.126753 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-72a62c5d-7f0e-4bb3-ab68-0debe481fed0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:32.127444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-72a62c5d-7f0e-4bb3-ab68-0debe481fed0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:32.127900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-72a62c5d-7f0e-4bb3-ab68-0debe481fed0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:32.128325 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-72a62c5d-7f0e-4bb3-ab68-0debe481fed0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:32.159089 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-72a62c5d-7f0e-4bb3-ab68-0debe481fed0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046098 Sep 08 14:00:32.159556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 535/961] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:32 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:33.171966 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-dd3f152b-b24f-4514-8a54-758d4185cb72 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:33.174548 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-dd3f152b-b24f-4514-8a54-758d4185cb72 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:33.181464 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd3f152b-b24f-4514-8a54-758d4185cb72 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:33.181951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd3f152b-b24f-4514-8a54-758d4185cb72 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:33.182598 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd3f152b-b24f-4514-8a54-758d4185cb72 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:33.183095 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd3f152b-b24f-4514-8a54-758d4185cb72 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:33.183536 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd3f152b-b24f-4514-8a54-758d4185cb72 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:33.184420 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd3f152b-b24f-4514-8a54-758d4185cb72 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:33.224485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-dd3f152b-b24f-4514-8a54-758d4185cb72 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.056744 Sep 08 14:00:33.225093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 427/962] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:33 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:34.237298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e01da501-eeaa-4dbd-a89a-1f355259806d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:34.239967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e01da501-eeaa-4dbd-a89a-1f355259806d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:34.246503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e01da501-eeaa-4dbd-a89a-1f355259806d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:34.247096 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e01da501-eeaa-4dbd-a89a-1f355259806d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:34.247542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e01da501-eeaa-4dbd-a89a-1f355259806d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:34.248238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e01da501-eeaa-4dbd-a89a-1f355259806d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:34.248725 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e01da501-eeaa-4dbd-a89a-1f355259806d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:34.249154 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e01da501-eeaa-4dbd-a89a-1f355259806d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:34.278923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e01da501-eeaa-4dbd-a89a-1f355259806d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045763 Sep 08 14:00:34.279471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 536/963] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:34 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:35.292735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9e8025f9-3983-4714-92b8-77976b258291 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:35.294843 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9e8025f9-3983-4714-92b8-77976b258291 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:35.297373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:00:35.297572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:00:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments (ip 10.222.0.25) !!! Sep 08 14:00:35.304428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e8025f9-3983-4714-92b8-77976b258291 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:35.304920 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e8025f9-3983-4714-92b8-77976b258291 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:35.305349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e8025f9-3983-4714-92b8-77976b258291 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:35.306056 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e8025f9-3983-4714-92b8-77976b258291 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:35.306526 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e8025f9-3983-4714-92b8-77976b258291 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:35.306938 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e8025f9-3983-4714-92b8-77976b258291 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:35.343048 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9e8025f9-3983-4714-92b8-77976b258291 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054342 Sep 08 14:00:35.343624 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 428/964] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:35 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:36.354993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8751c47f-1eca-4f55-8a6c-19a5ec4451df tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:36.357184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8751c47f-1eca-4f55-8a6c-19a5ec4451df tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:36.364238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8751c47f-1eca-4f55-8a6c-19a5ec4451df tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:36.364727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8751c47f-1eca-4f55-8a6c-19a5ec4451df tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:36.365166 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8751c47f-1eca-4f55-8a6c-19a5ec4451df tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:36.365858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8751c47f-1eca-4f55-8a6c-19a5ec4451df tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:36.366353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8751c47f-1eca-4f55-8a6c-19a5ec4451df tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:36.366784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8751c47f-1eca-4f55-8a6c-19a5ec4451df tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:36.396663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8751c47f-1eca-4f55-8a6c-19a5ec4451df tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045680 Sep 08 14:00:36.397211 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 537/965] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:36 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:37.409066 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3e5380fd-32f4-412c-bc92-8791f869e66a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:37.411209 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3e5380fd-32f4-412c-bc92-8791f869e66a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:37.417518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e5380fd-32f4-412c-bc92-8791f869e66a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:37.417996 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e5380fd-32f4-412c-bc92-8791f869e66a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:37.418886 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e5380fd-32f4-412c-bc92-8791f869e66a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:37.419213 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e5380fd-32f4-412c-bc92-8791f869e66a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:37.419659 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e5380fd-32f4-412c-bc92-8791f869e66a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:37.420076 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e5380fd-32f4-412c-bc92-8791f869e66a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:37.448660 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3e5380fd-32f4-412c-bc92-8791f869e66a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043655 Sep 08 14:00:37.449204 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 429/966] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:37 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:38.462804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-51140181-393a-40ef-855b-68be06501bd9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:38.464989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-51140181-393a-40ef-855b-68be06501bd9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:38.473190 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51140181-393a-40ef-855b-68be06501bd9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:38.473968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51140181-393a-40ef-855b-68be06501bd9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:38.474648 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51140181-393a-40ef-855b-68be06501bd9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:38.475379 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51140181-393a-40ef-855b-68be06501bd9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:38.475844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51140181-393a-40ef-855b-68be06501bd9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:38.476278 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51140181-393a-40ef-855b-68be06501bd9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:38.508328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-51140181-393a-40ef-855b-68be06501bd9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049520 Sep 08 14:00:38.508882 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 538/967] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:38 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:39.520643 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8fc83a41-db3e-4650-8478-3504bb9258a2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:39.523839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8fc83a41-db3e-4650-8478-3504bb9258a2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:39.529381 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fc83a41-db3e-4650-8478-3504bb9258a2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:39.529858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fc83a41-db3e-4650-8478-3504bb9258a2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:39.530341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fc83a41-db3e-4650-8478-3504bb9258a2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:39.531022 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fc83a41-db3e-4650-8478-3504bb9258a2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:39.531469 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fc83a41-db3e-4650-8478-3504bb9258a2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:39.531887 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fc83a41-db3e-4650-8478-3504bb9258a2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:39.564989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8fc83a41-db3e-4650-8478-3504bb9258a2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048545 Sep 08 14:00:39.565567 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 430/968] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:39 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:40.577093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1f3a7a4a-0ac6-4bea-93e6-5dd24038a126 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:40.579246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1f3a7a4a-0ac6-4bea-93e6-5dd24038a126 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:40.586570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1f3a7a4a-0ac6-4bea-93e6-5dd24038a126 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:40.587362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1f3a7a4a-0ac6-4bea-93e6-5dd24038a126 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:40.587863 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1f3a7a4a-0ac6-4bea-93e6-5dd24038a126 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:40.588573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1f3a7a4a-0ac6-4bea-93e6-5dd24038a126 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:40.589034 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1f3a7a4a-0ac6-4bea-93e6-5dd24038a126 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:40.589475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1f3a7a4a-0ac6-4bea-93e6-5dd24038a126 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:40.620155 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1f3a7a4a-0ac6-4bea-93e6-5dd24038a126 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047143 Sep 08 14:00:40.620717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 539/969] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:40 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:40.800764 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5e7f4405-98a2-443d-b72d-51c9298d8747 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:40.801351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5e7f4405-98a2-443d-b72d-51c9298d8747 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:40.807882 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e7f4405-98a2-443d-b72d-51c9298d8747 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:40.808518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e7f4405-98a2-443d-b72d-51c9298d8747 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:40.808950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e7f4405-98a2-443d-b72d-51c9298d8747 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:40.809643 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e7f4405-98a2-443d-b72d-51c9298d8747 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:40.810090 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e7f4405-98a2-443d-b72d-51c9298d8747 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:40.810532 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e7f4405-98a2-443d-b72d-51c9298d8747 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:40.911835 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-5e7f4405-98a2-443d-b72d-51c9298d8747 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lazy-loading 'services' on Instance uuid 19db4aa2-f574-43fa-bbf8-c3042b243152 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:00:41.132460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5e7f4405-98a2-443d-b72d-51c9298d8747 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments/3d6a13a3-142c-49d7-9ef4-0c61f151b4c3" status: 202 len: 0 microversion: 2.1 time: 0.335938 Sep 08 14:00:41.133025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 431/970] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:00:40 2022] DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments/3d6a13a3-142c-49d7-9ef4-0c61f151b4c3 => generated 0 bytes in 338 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:00:41.142094 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-01b03699-f372-43d8-8b6c-1dd4eeb85a2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:41.147417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-01b03699-f372-43d8-8b6c-1dd4eeb85a2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:41.155934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01b03699-f372-43d8-8b6c-1dd4eeb85a2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:41.156657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01b03699-f372-43d8-8b6c-1dd4eeb85a2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:41.157064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01b03699-f372-43d8-8b6c-1dd4eeb85a2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:41.157760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01b03699-f372-43d8-8b6c-1dd4eeb85a2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:41.159048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01b03699-f372-43d8-8b6c-1dd4eeb85a2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:41.159559 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01b03699-f372-43d8-8b6c-1dd4eeb85a2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:41.194866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-01b03699-f372-43d8-8b6c-1dd4eeb85a2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057391 Sep 08 14:00:41.195539 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 540/971] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:00:41 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:41.635486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a82fe57e-8b88-47ba-a49d-f0edb3e348af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:41.637891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a82fe57e-8b88-47ba-a49d-f0edb3e348af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:41.640551 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=36 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:41.646162 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a82fe57e-8b88-47ba-a49d-f0edb3e348af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:41.646948 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a82fe57e-8b88-47ba-a49d-f0edb3e348af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:41.647402 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a82fe57e-8b88-47ba-a49d-f0edb3e348af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:41.648119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a82fe57e-8b88-47ba-a49d-f0edb3e348af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:41.648582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a82fe57e-8b88-47ba-a49d-f0edb3e348af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:41.649006 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a82fe57e-8b88-47ba-a49d-f0edb3e348af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:41.685968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a82fe57e-8b88-47ba-a49d-f0edb3e348af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055001 Sep 08 14:00:41.686600 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 432/972] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:41 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:42.207974 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-929d0336-ae71-4bfd-9370-ef97a59741c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:42.210278 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-929d0336-ae71-4bfd-9370-ef97a59741c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:42.223170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-929d0336-ae71-4bfd-9370-ef97a59741c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:42.223675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-929d0336-ae71-4bfd-9370-ef97a59741c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:42.224122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-929d0336-ae71-4bfd-9370-ef97a59741c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:42.224829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-929d0336-ae71-4bfd-9370-ef97a59741c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:42.225323 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-929d0336-ae71-4bfd-9370-ef97a59741c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:42.225754 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-929d0336-ae71-4bfd-9370-ef97a59741c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:42.257744 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-929d0336-ae71-4bfd-9370-ef97a59741c4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053750 Sep 08 14:00:42.258343 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 541/973] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:00:42 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:42.699286 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-77d5f73e-42c4-4dea-bde3-824b30d89f6a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:42.701408 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-77d5f73e-42c4-4dea-bde3-824b30d89f6a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:42.709095 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77d5f73e-42c4-4dea-bde3-824b30d89f6a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:42.709582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77d5f73e-42c4-4dea-bde3-824b30d89f6a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:42.710018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77d5f73e-42c4-4dea-bde3-824b30d89f6a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:42.710742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77d5f73e-42c4-4dea-bde3-824b30d89f6a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:42.711193 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77d5f73e-42c4-4dea-bde3-824b30d89f6a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:42.711618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77d5f73e-42c4-4dea-bde3-824b30d89f6a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:42.740465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-77d5f73e-42c4-4dea-bde3-824b30d89f6a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045553 Sep 08 14:00:42.741028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 433/974] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:42 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:43.271364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9382f88e-1f3e-4a7e-aa2b-7a2ce0ab9dda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:43.274816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9382f88e-1f3e-4a7e-aa2b-7a2ce0ab9dda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:43.286460 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9382f88e-1f3e-4a7e-aa2b-7a2ce0ab9dda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:43.287297 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9382f88e-1f3e-4a7e-aa2b-7a2ce0ab9dda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:43.287859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9382f88e-1f3e-4a7e-aa2b-7a2ce0ab9dda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:43.288662 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9382f88e-1f3e-4a7e-aa2b-7a2ce0ab9dda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:43.289175 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9382f88e-1f3e-4a7e-aa2b-7a2ce0ab9dda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:43.289599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9382f88e-1f3e-4a7e-aa2b-7a2ce0ab9dda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:43.328709 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9382f88e-1f3e-4a7e-aa2b-7a2ce0ab9dda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061601 Sep 08 14:00:43.329296 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 542/975] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:00:43 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:43.752668 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bb14737f-f9ed-472e-9fb1-646a594be3cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:43.754900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bb14737f-f9ed-472e-9fb1-646a594be3cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:43.762080 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bb14737f-f9ed-472e-9fb1-646a594be3cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:43.762596 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bb14737f-f9ed-472e-9fb1-646a594be3cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:43.763030 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bb14737f-f9ed-472e-9fb1-646a594be3cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:43.763733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bb14737f-f9ed-472e-9fb1-646a594be3cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:43.764184 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bb14737f-f9ed-472e-9fb1-646a594be3cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:43.764608 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bb14737f-f9ed-472e-9fb1-646a594be3cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:43.800444 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bb14737f-f9ed-472e-9fb1-646a594be3cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051794 Sep 08 14:00:43.801028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 434/976] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:43 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:00:44.342874 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fc88597e-cee2-4706-9034-218e34e59dc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:44.345295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fc88597e-cee2-4706-9034-218e34e59dc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:44.356534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc88597e-cee2-4706-9034-218e34e59dc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:44.357403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc88597e-cee2-4706-9034-218e34e59dc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:44.361272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc88597e-cee2-4706-9034-218e34e59dc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:44.361272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc88597e-cee2-4706-9034-218e34e59dc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:44.361272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc88597e-cee2-4706-9034-218e34e59dc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:44.361272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc88597e-cee2-4706-9034-218e34e59dc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:44.368484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:44.406806 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fc88597e-cee2-4706-9034-218e34e59dc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.068193 Sep 08 14:00:44.407403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 543/977] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:00:44 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:44.812820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3a3483c0-7507-42a0-a432-6e16e4e69a6e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:44.815508 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3a3483c0-7507-42a0-a432-6e16e4e69a6e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:44.822190 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3a3483c0-7507-42a0-a432-6e16e4e69a6e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:44.822697 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3a3483c0-7507-42a0-a432-6e16e4e69a6e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:44.823117 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3a3483c0-7507-42a0-a432-6e16e4e69a6e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:44.823820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3a3483c0-7507-42a0-a432-6e16e4e69a6e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:44.824264 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3a3483c0-7507-42a0-a432-6e16e4e69a6e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:44.824689 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3a3483c0-7507-42a0-a432-6e16e4e69a6e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:44.854496 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3a3483c0-7507-42a0-a432-6e16e4e69a6e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045635 Sep 08 14:00:44.855051 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 435/978] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:44 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:45.420945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-86c086c1-65ac-4d91-82c6-260e5a4346e8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:45.423671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-86c086c1-65ac-4d91-82c6-260e5a4346e8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:45.430718 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-86c086c1-65ac-4d91-82c6-260e5a4346e8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:45.431215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-86c086c1-65ac-4d91-82c6-260e5a4346e8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:45.431662 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-86c086c1-65ac-4d91-82c6-260e5a4346e8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:45.432374 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-86c086c1-65ac-4d91-82c6-260e5a4346e8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:45.432837 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-86c086c1-65ac-4d91-82c6-260e5a4346e8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:45.433267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-86c086c1-65ac-4d91-82c6-260e5a4346e8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:45.468517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-86c086c1-65ac-4d91-82c6-260e5a4346e8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051811 Sep 08 14:00:45.469086 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 544/979] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:00:45 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:45.868101 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bf60590d-86d8-4b4b-b567-3b6112c64cfc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:45.870548 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bf60590d-86d8-4b4b-b567-3b6112c64cfc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:45.877233 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf60590d-86d8-4b4b-b567-3b6112c64cfc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:45.877716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf60590d-86d8-4b4b-b567-3b6112c64cfc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:45.878169 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf60590d-86d8-4b4b-b567-3b6112c64cfc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:45.878864 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf60590d-86d8-4b4b-b567-3b6112c64cfc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:45.879307 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf60590d-86d8-4b4b-b567-3b6112c64cfc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:45.879726 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf60590d-86d8-4b4b-b567-3b6112c64cfc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:45.913701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bf60590d-86d8-4b4b-b567-3b6112c64cfc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049889 Sep 08 14:00:45.914191 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 436/980] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:45 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:46.482259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3ec34a2d-ad28-4b4b-a76e-ad4855b13122 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:46.484464 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3ec34a2d-ad28-4b4b-a76e-ad4855b13122 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:46.491935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3ec34a2d-ad28-4b4b-a76e-ad4855b13122 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:46.492434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3ec34a2d-ad28-4b4b-a76e-ad4855b13122 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:46.492877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3ec34a2d-ad28-4b4b-a76e-ad4855b13122 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:46.494032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3ec34a2d-ad28-4b4b-a76e-ad4855b13122 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:46.494032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3ec34a2d-ad28-4b4b-a76e-ad4855b13122 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:46.494684 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3ec34a2d-ad28-4b4b-a76e-ad4855b13122 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:46.531226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3ec34a2d-ad28-4b4b-a76e-ad4855b13122 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053230 Sep 08 14:00:46.531804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 545/981] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:00:46 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:46.928620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-33bbb2af-ff05-4d2f-bc7b-9e2b408cdff8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:46.931628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-33bbb2af-ff05-4d2f-bc7b-9e2b408cdff8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:46.940464 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33bbb2af-ff05-4d2f-bc7b-9e2b408cdff8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:46.940944 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33bbb2af-ff05-4d2f-bc7b-9e2b408cdff8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:46.941481 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33bbb2af-ff05-4d2f-bc7b-9e2b408cdff8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:46.942664 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33bbb2af-ff05-4d2f-bc7b-9e2b408cdff8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:46.943429 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33bbb2af-ff05-4d2f-bc7b-9e2b408cdff8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:46.944039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33bbb2af-ff05-4d2f-bc7b-9e2b408cdff8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:46.978772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-33bbb2af-ff05-4d2f-bc7b-9e2b408cdff8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054247 Sep 08 14:00:46.979344 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 437/982] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:46 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:00:47.545859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a54594b1-b5ee-4a47-adfe-d2c0bd811f5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:47.548282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a54594b1-b5ee-4a47-adfe-d2c0bd811f5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:47.556369 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a54594b1-b5ee-4a47-adfe-d2c0bd811f5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:47.559927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a54594b1-b5ee-4a47-adfe-d2c0bd811f5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:47.559927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a54594b1-b5ee-4a47-adfe-d2c0bd811f5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:47.559927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a54594b1-b5ee-4a47-adfe-d2c0bd811f5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:47.559927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a54594b1-b5ee-4a47-adfe-d2c0bd811f5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:47.560405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a54594b1-b5ee-4a47-adfe-d2c0bd811f5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:47.590198 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a54594b1-b5ee-4a47-adfe-d2c0bd811f5f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049357 Sep 08 14:00:47.590765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 546/983] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:00:47 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:47.992038 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-16537e87-de94-47e1-8f0b-5f69c98a8bc4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:47.994292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-16537e87-de94-47e1-8f0b-5f69c98a8bc4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:48.001760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-16537e87-de94-47e1-8f0b-5f69c98a8bc4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:48.002381 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-16537e87-de94-47e1-8f0b-5f69c98a8bc4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:48.002821 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-16537e87-de94-47e1-8f0b-5f69c98a8bc4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:48.003545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-16537e87-de94-47e1-8f0b-5f69c98a8bc4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:48.004003 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-16537e87-de94-47e1-8f0b-5f69c98a8bc4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:48.004443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-16537e87-de94-47e1-8f0b-5f69c98a8bc4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:48.023196 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:48.037367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-16537e87-de94-47e1-8f0b-5f69c98a8bc4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049717 Sep 08 14:00:48.038170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 438/984] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:47 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:48.603886 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8c8333cf-d7c0-45e2-a35f-fbabb808554c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:48.606104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8c8333cf-d7c0-45e2-a35f-fbabb808554c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:48.613438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c8333cf-d7c0-45e2-a35f-fbabb808554c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:48.613849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c8333cf-d7c0-45e2-a35f-fbabb808554c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:48.614328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c8333cf-d7c0-45e2-a35f-fbabb808554c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:48.615033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c8333cf-d7c0-45e2-a35f-fbabb808554c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:48.615490 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c8333cf-d7c0-45e2-a35f-fbabb808554c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:48.615925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c8333cf-d7c0-45e2-a35f-fbabb808554c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:48.652663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8c8333cf-d7c0-45e2-a35f-fbabb808554c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053006 Sep 08 14:00:48.653687 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 547/985] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:00:48 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:49.050579 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6dc36b81-70a7-4c5f-b086-123e480395db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:49.052645 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6dc36b81-70a7-4c5f-b086-123e480395db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:49.059764 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6dc36b81-70a7-4c5f-b086-123e480395db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:49.060247 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6dc36b81-70a7-4c5f-b086-123e480395db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:49.060682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6dc36b81-70a7-4c5f-b086-123e480395db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:49.061384 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6dc36b81-70a7-4c5f-b086-123e480395db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:49.061835 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6dc36b81-70a7-4c5f-b086-123e480395db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:49.062296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6dc36b81-70a7-4c5f-b086-123e480395db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:49.098170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6dc36b81-70a7-4c5f-b086-123e480395db tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052682 Sep 08 14:00:49.098765 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 439/986] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:49 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:49.668283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1155d1d0-9599-4313-911b-78142eac6d68 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:49.670489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1155d1d0-9599-4313-911b-78142eac6d68 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:49.678777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1155d1d0-9599-4313-911b-78142eac6d68 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:49.679273 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1155d1d0-9599-4313-911b-78142eac6d68 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:49.679860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1155d1d0-9599-4313-911b-78142eac6d68 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:49.681082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1155d1d0-9599-4313-911b-78142eac6d68 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:49.681913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1155d1d0-9599-4313-911b-78142eac6d68 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:49.682721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1155d1d0-9599-4313-911b-78142eac6d68 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:49.722190 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1155d1d0-9599-4313-911b-78142eac6d68 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059526 Sep 08 14:00:49.722768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 548/987] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:00:49 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:50.112499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c3b35390-cb8b-4e06-9539-1e12a96487e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:50.114646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c3b35390-cb8b-4e06-9539-1e12a96487e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:50.121154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c3b35390-cb8b-4e06-9539-1e12a96487e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:50.121633 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c3b35390-cb8b-4e06-9539-1e12a96487e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:50.122064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c3b35390-cb8b-4e06-9539-1e12a96487e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:50.122777 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c3b35390-cb8b-4e06-9539-1e12a96487e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:50.123227 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c3b35390-cb8b-4e06-9539-1e12a96487e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:50.123645 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c3b35390-cb8b-4e06-9539-1e12a96487e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:50.153271 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c3b35390-cb8b-4e06-9539-1e12a96487e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045348 Sep 08 14:00:50.153824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 440/988] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:50 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:50.735072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-32492297-eb4c-415a-81dc-464eb34e450b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:50.737146 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-32492297-eb4c-415a-81dc-464eb34e450b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:50.743619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32492297-eb4c-415a-81dc-464eb34e450b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:50.744108 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32492297-eb4c-415a-81dc-464eb34e450b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:50.744546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32492297-eb4c-415a-81dc-464eb34e450b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:50.745238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32492297-eb4c-415a-81dc-464eb34e450b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:50.745721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32492297-eb4c-415a-81dc-464eb34e450b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:50.746165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32492297-eb4c-415a-81dc-464eb34e450b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:50.776280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-32492297-eb4c-415a-81dc-464eb34e450b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045219 Sep 08 14:00:50.776829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 549/989] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:00:50 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:51.166772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e9177e9b-404f-4676-bc1c-fbcd59703006 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:51.168875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e9177e9b-404f-4676-bc1c-fbcd59703006 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:51.175807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e9177e9b-404f-4676-bc1c-fbcd59703006 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:51.176281 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e9177e9b-404f-4676-bc1c-fbcd59703006 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:51.176710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e9177e9b-404f-4676-bc1c-fbcd59703006 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:51.177387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e9177e9b-404f-4676-bc1c-fbcd59703006 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:51.177832 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e9177e9b-404f-4676-bc1c-fbcd59703006 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:51.178277 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e9177e9b-404f-4676-bc1c-fbcd59703006 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:51.207295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e9177e9b-404f-4676-bc1c-fbcd59703006 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044618 Sep 08 14:00:51.207844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 441/990] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:51 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:51.789984 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ed917282-eba8-4b6b-95c5-c5b26ad8b9d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:51.792126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ed917282-eba8-4b6b-95c5-c5b26ad8b9d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:51.799160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed917282-eba8-4b6b-95c5-c5b26ad8b9d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:51.799652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed917282-eba8-4b6b-95c5-c5b26ad8b9d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:51.800095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed917282-eba8-4b6b-95c5-c5b26ad8b9d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:51.800788 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed917282-eba8-4b6b-95c5-c5b26ad8b9d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:51.801242 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed917282-eba8-4b6b-95c5-c5b26ad8b9d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:51.801670 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed917282-eba8-4b6b-95c5-c5b26ad8b9d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:51.830553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ed917282-eba8-4b6b-95c5-c5b26ad8b9d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044799 Sep 08 14:00:51.831154 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 550/991] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:00:51 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:52.220669 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8bea5703-9f53-4837-8b8a-8f19611eb3c9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:52.222847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8bea5703-9f53-4837-8b8a-8f19611eb3c9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:52.229319 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8bea5703-9f53-4837-8b8a-8f19611eb3c9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:52.229795 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8bea5703-9f53-4837-8b8a-8f19611eb3c9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:52.230254 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8bea5703-9f53-4837-8b8a-8f19611eb3c9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:52.230944 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8bea5703-9f53-4837-8b8a-8f19611eb3c9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:52.231403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8bea5703-9f53-4837-8b8a-8f19611eb3c9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:52.231804 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8bea5703-9f53-4837-8b8a-8f19611eb3c9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:52.260439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8bea5703-9f53-4837-8b8a-8f19611eb3c9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044286 Sep 08 14:00:52.261002 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 442/992] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:52 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:52.843827 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c17f7ee8-2bf1-49fd-b3c2-019a34ca1c2c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:52.845997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c17f7ee8-2bf1-49fd-b3c2-019a34ca1c2c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:52.852551 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c17f7ee8-2bf1-49fd-b3c2-019a34ca1c2c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:52.853024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c17f7ee8-2bf1-49fd-b3c2-019a34ca1c2c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:52.853464 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c17f7ee8-2bf1-49fd-b3c2-019a34ca1c2c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:52.854182 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c17f7ee8-2bf1-49fd-b3c2-019a34ca1c2c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:52.854643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c17f7ee8-2bf1-49fd-b3c2-019a34ca1c2c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:52.855068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c17f7ee8-2bf1-49fd-b3c2-019a34ca1c2c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:52.885951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c17f7ee8-2bf1-49fd-b3c2-019a34ca1c2c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046146 Sep 08 14:00:52.886547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 551/993] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:00:52 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:53.274184 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-23356370-47aa-446c-9bd4-22d0d3c37c97 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:53.276266 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-23356370-47aa-446c-9bd4-22d0d3c37c97 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:53.283480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-23356370-47aa-446c-9bd4-22d0d3c37c97 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:53.283951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-23356370-47aa-446c-9bd4-22d0d3c37c97 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:53.284375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-23356370-47aa-446c-9bd4-22d0d3c37c97 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:53.285056 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-23356370-47aa-446c-9bd4-22d0d3c37c97 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:53.285504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-23356370-47aa-446c-9bd4-22d0d3c37c97 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:53.286006 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-23356370-47aa-446c-9bd4-22d0d3c37c97 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:53.316176 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-23356370-47aa-446c-9bd4-22d0d3c37c97 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046176 Sep 08 14:00:53.316737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 443/994] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:53 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:53.898873 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-38d46f64-218a-4ce4-bb0c-c9a45bc30482 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:53.901319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-38d46f64-218a-4ce4-bb0c-c9a45bc30482 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:53.903712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=44 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:53.908866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38d46f64-218a-4ce4-bb0c-c9a45bc30482 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:53.909361 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38d46f64-218a-4ce4-bb0c-c9a45bc30482 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:53.909802 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38d46f64-218a-4ce4-bb0c-c9a45bc30482 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:53.910558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38d46f64-218a-4ce4-bb0c-c9a45bc30482 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:53.911009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38d46f64-218a-4ce4-bb0c-c9a45bc30482 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:53.911441 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38d46f64-218a-4ce4-bb0c-c9a45bc30482 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:53.942200 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-38d46f64-218a-4ce4-bb0c-c9a45bc30482 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047227 Sep 08 14:00:53.942761 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 552/995] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:00:53 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:54.328434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f8142504-d8eb-4cc7-9a53-043edd63ed6d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:54.330639 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f8142504-d8eb-4cc7-9a53-043edd63ed6d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:54.337912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f8142504-d8eb-4cc7-9a53-043edd63ed6d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:54.338427 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f8142504-d8eb-4cc7-9a53-043edd63ed6d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:54.338859 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f8142504-d8eb-4cc7-9a53-043edd63ed6d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:54.339550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f8142504-d8eb-4cc7-9a53-043edd63ed6d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:54.340063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f8142504-d8eb-4cc7-9a53-043edd63ed6d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:54.340477 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f8142504-d8eb-4cc7-9a53-043edd63ed6d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:54.371036 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f8142504-d8eb-4cc7-9a53-043edd63ed6d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046533 Sep 08 14:00:54.371612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 444/996] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:54 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:54.955852 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-802fe62d-c623-4f04-9fc5-6434de2d58e2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:54.957928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-802fe62d-c623-4f04-9fc5-6434de2d58e2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:54.964512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-802fe62d-c623-4f04-9fc5-6434de2d58e2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:54.965001 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-802fe62d-c623-4f04-9fc5-6434de2d58e2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:54.965449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-802fe62d-c623-4f04-9fc5-6434de2d58e2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:54.966170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-802fe62d-c623-4f04-9fc5-6434de2d58e2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:54.966642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-802fe62d-c623-4f04-9fc5-6434de2d58e2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:54.967074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-802fe62d-c623-4f04-9fc5-6434de2d58e2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:54.997989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-802fe62d-c623-4f04-9fc5-6434de2d58e2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047152 Sep 08 14:00:54.998621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 553/997] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:00:54 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:55.384385 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bf0f2dbe-8739-4c69-a201-77675229c438 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:55.386547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bf0f2dbe-8739-4c69-a201-77675229c438 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:55.394163 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf0f2dbe-8739-4c69-a201-77675229c438 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:55.394660 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf0f2dbe-8739-4c69-a201-77675229c438 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:55.395084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf0f2dbe-8739-4c69-a201-77675229c438 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:55.395768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf0f2dbe-8739-4c69-a201-77675229c438 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:55.396216 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf0f2dbe-8739-4c69-a201-77675229c438 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:55.396628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf0f2dbe-8739-4c69-a201-77675229c438 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:55.425797 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bf0f2dbe-8739-4c69-a201-77675229c438 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045479 Sep 08 14:00:55.426380 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 445/998] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:55 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:56.010582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7b90fc6e-06b1-458a-9c8d-e304f03157ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:56.012764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7b90fc6e-06b1-458a-9c8d-e304f03157ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:56.019498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b90fc6e-06b1-458a-9c8d-e304f03157ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:56.019992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b90fc6e-06b1-458a-9c8d-e304f03157ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:56.020437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b90fc6e-06b1-458a-9c8d-e304f03157ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:56.021132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b90fc6e-06b1-458a-9c8d-e304f03157ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:56.021592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b90fc6e-06b1-458a-9c8d-e304f03157ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:56.022021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b90fc6e-06b1-458a-9c8d-e304f03157ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:56.052484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7b90fc6e-06b1-458a-9c8d-e304f03157ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045910 Sep 08 14:00:56.053911 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 554/999] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:00:56 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:56.438687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-de999c49-3b1f-4d7c-ac40-be4f5b1d4654 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:56.440860 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-de999c49-3b1f-4d7c-ac40-be4f5b1d4654 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:56.448303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-de999c49-3b1f-4d7c-ac40-be4f5b1d4654 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:56.448798 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-de999c49-3b1f-4d7c-ac40-be4f5b1d4654 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:56.449230 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-de999c49-3b1f-4d7c-ac40-be4f5b1d4654 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:56.449992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-de999c49-3b1f-4d7c-ac40-be4f5b1d4654 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:56.450423 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-de999c49-3b1f-4d7c-ac40-be4f5b1d4654 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:56.450847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-de999c49-3b1f-4d7c-ac40-be4f5b1d4654 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:56.481885 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-de999c49-3b1f-4d7c-ac40-be4f5b1d4654 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047253 Sep 08 14:00:56.482493 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 446/1000] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:56 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:00:57.065729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9afe4021-e778-4254-b023-614d302e80a5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:57.067824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9afe4021-e778-4254-b023-614d302e80a5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:57.074348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9afe4021-e778-4254-b023-614d302e80a5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:57.074837 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9afe4021-e778-4254-b023-614d302e80a5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:57.075278 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9afe4021-e778-4254-b023-614d302e80a5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:57.075968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9afe4021-e778-4254-b023-614d302e80a5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:57.076422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9afe4021-e778-4254-b023-614d302e80a5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:57.077429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9afe4021-e778-4254-b023-614d302e80a5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:57.106317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9afe4021-e778-4254-b023-614d302e80a5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045081 Sep 08 14:00:57.106865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 555/1001] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:00:57 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:57.494582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5303f384-3666-47c8-927f-c8671c3981b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:57.496748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5303f384-3666-47c8-927f-c8671c3981b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:57.504226 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5303f384-3666-47c8-927f-c8671c3981b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:57.504703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5303f384-3666-47c8-927f-c8671c3981b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:57.505129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5303f384-3666-47c8-927f-c8671c3981b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:57.505819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5303f384-3666-47c8-927f-c8671c3981b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:57.506304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5303f384-3666-47c8-927f-c8671c3981b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:57.506723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5303f384-3666-47c8-927f-c8671c3981b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:57.543705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5303f384-3666-47c8-927f-c8671c3981b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053143 Sep 08 14:00:57.544288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 447/1002] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:57 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:00:58.125119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0b42eaaf-3a69-4f26-91f8-36045c28b0ac tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:58.127365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0b42eaaf-3a69-4f26-91f8-36045c28b0ac tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:58.133596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b42eaaf-3a69-4f26-91f8-36045c28b0ac tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:58.134075 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b42eaaf-3a69-4f26-91f8-36045c28b0ac tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:58.134542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b42eaaf-3a69-4f26-91f8-36045c28b0ac tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:58.135243 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b42eaaf-3a69-4f26-91f8-36045c28b0ac tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:58.135694 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b42eaaf-3a69-4f26-91f8-36045c28b0ac tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:58.136116 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b42eaaf-3a69-4f26-91f8-36045c28b0ac tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:58.166545 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0b42eaaf-3a69-4f26-91f8-36045c28b0ac tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045259 Sep 08 14:00:58.167093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 556/1003] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:00:58 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:58.558511 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-dd4a72d8-28d7-4047-a95f-82e7c15df26b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:58.563461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-dd4a72d8-28d7-4047-a95f-82e7c15df26b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:58.579468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd4a72d8-28d7-4047-a95f-82e7c15df26b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:58.580073 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd4a72d8-28d7-4047-a95f-82e7c15df26b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:58.580572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd4a72d8-28d7-4047-a95f-82e7c15df26b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:58.581311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd4a72d8-28d7-4047-a95f-82e7c15df26b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:58.584369 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd4a72d8-28d7-4047-a95f-82e7c15df26b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:58.584940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd4a72d8-28d7-4047-a95f-82e7c15df26b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:58.618398 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-dd4a72d8-28d7-4047-a95f-82e7c15df26b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.064290 Sep 08 14:00:58.618974 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 448/1004] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:58 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:00:59.182410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a270831e-0ada-4d30-8f7f-05f95d870139 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:59.184555 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a270831e-0ada-4d30-8f7f-05f95d870139 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:59.191193 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a270831e-0ada-4d30-8f7f-05f95d870139 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:59.191708 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a270831e-0ada-4d30-8f7f-05f95d870139 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:59.192147 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a270831e-0ada-4d30-8f7f-05f95d870139 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:59.192847 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a270831e-0ada-4d30-8f7f-05f95d870139 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:59.193297 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a270831e-0ada-4d30-8f7f-05f95d870139 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:59.193720 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a270831e-0ada-4d30-8f7f-05f95d870139 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:59.225824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a270831e-0ada-4d30-8f7f-05f95d870139 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049066 Sep 08 14:00:59.226427 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 557/1005] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:00:59 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:00:59.634585 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-66803573-f0d7-4eb6-a3c7-5ce31fc25fda tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:00:59.636762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-66803573-f0d7-4eb6-a3c7-5ce31fc25fda tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:00:59.644004 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-66803573-f0d7-4eb6-a3c7-5ce31fc25fda tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:59.644487 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-66803573-f0d7-4eb6-a3c7-5ce31fc25fda tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:59.644920 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-66803573-f0d7-4eb6-a3c7-5ce31fc25fda tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:59.645618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-66803573-f0d7-4eb6-a3c7-5ce31fc25fda tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:00:59.646063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-66803573-f0d7-4eb6-a3c7-5ce31fc25fda tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:00:59.646515 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-66803573-f0d7-4eb6-a3c7-5ce31fc25fda tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:00:59.692004 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-66803573-f0d7-4eb6-a3c7-5ce31fc25fda tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061443 Sep 08 14:00:59.692600 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 449/1006] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:00:59 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:00.239227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-924f4798-ca1f-4112-8f9f-14cb88e9bc99 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:00.241988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-924f4798-ca1f-4112-8f9f-14cb88e9bc99 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:00.250216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-924f4798-ca1f-4112-8f9f-14cb88e9bc99 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:00.250720 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-924f4798-ca1f-4112-8f9f-14cb88e9bc99 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:00.251166 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-924f4798-ca1f-4112-8f9f-14cb88e9bc99 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:00.251889 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-924f4798-ca1f-4112-8f9f-14cb88e9bc99 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:00.252346 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-924f4798-ca1f-4112-8f9f-14cb88e9bc99 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:00.252775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-924f4798-ca1f-4112-8f9f-14cb88e9bc99 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:00.285573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-924f4798-ca1f-4112-8f9f-14cb88e9bc99 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050500 Sep 08 14:01:00.286369 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 558/1007] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:00 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:00.705038 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cf48f110-00b3-42fb-a284-8e30bca3f9a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:00.707326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cf48f110-00b3-42fb-a284-8e30bca3f9a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:00.714249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cf48f110-00b3-42fb-a284-8e30bca3f9a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:00.714742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cf48f110-00b3-42fb-a284-8e30bca3f9a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:00.715174 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cf48f110-00b3-42fb-a284-8e30bca3f9a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:00.715946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cf48f110-00b3-42fb-a284-8e30bca3f9a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:00.716436 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cf48f110-00b3-42fb-a284-8e30bca3f9a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:00.717366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cf48f110-00b3-42fb-a284-8e30bca3f9a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:00.750446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cf48f110-00b3-42fb-a284-8e30bca3f9a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049111 Sep 08 14:01:00.751057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 450/1008] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:00 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:01.298436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ffdc6235-56f7-468a-9b20-989cfe341fee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:01.300587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ffdc6235-56f7-468a-9b20-989cfe341fee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:01.306985 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ffdc6235-56f7-468a-9b20-989cfe341fee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:01.307472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ffdc6235-56f7-468a-9b20-989cfe341fee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:01.307912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ffdc6235-56f7-468a-9b20-989cfe341fee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:01.308604 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ffdc6235-56f7-468a-9b20-989cfe341fee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:01.309069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ffdc6235-56f7-468a-9b20-989cfe341fee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:01.309585 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ffdc6235-56f7-468a-9b20-989cfe341fee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:01.338474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:01.343765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ffdc6235-56f7-468a-9b20-989cfe341fee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049388 Sep 08 14:01:01.344383 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 559/1009] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:01 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:01.764345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7a28eeca-4eff-45bd-a36e-12e6e0ac7538 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:01.766682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7a28eeca-4eff-45bd-a36e-12e6e0ac7538 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:01.774361 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a28eeca-4eff-45bd-a36e-12e6e0ac7538 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:01.774846 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a28eeca-4eff-45bd-a36e-12e6e0ac7538 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:01.775282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a28eeca-4eff-45bd-a36e-12e6e0ac7538 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:01.775983 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a28eeca-4eff-45bd-a36e-12e6e0ac7538 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:01.776438 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a28eeca-4eff-45bd-a36e-12e6e0ac7538 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:01.776863 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a28eeca-4eff-45bd-a36e-12e6e0ac7538 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:01.808776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7a28eeca-4eff-45bd-a36e-12e6e0ac7538 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048972 Sep 08 14:01:01.809341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 451/1010] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:01 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:02.357785 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e2413553-5d3b-4ff2-8d26-861df3bf6db4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:02.360009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e2413553-5d3b-4ff2-8d26-861df3bf6db4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:02.367316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e2413553-5d3b-4ff2-8d26-861df3bf6db4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:02.367660 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e2413553-5d3b-4ff2-8d26-861df3bf6db4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:02.368088 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e2413553-5d3b-4ff2-8d26-861df3bf6db4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:02.368786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e2413553-5d3b-4ff2-8d26-861df3bf6db4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:02.369232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e2413553-5d3b-4ff2-8d26-861df3bf6db4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:02.369650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e2413553-5d3b-4ff2-8d26-861df3bf6db4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:02.400734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e2413553-5d3b-4ff2-8d26-861df3bf6db4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047669 Sep 08 14:01:02.401317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 560/1011] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:02 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:02.822067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c55ad565-301a-48d8-8f69-8c4a83eafc53 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:02.824212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c55ad565-301a-48d8-8f69-8c4a83eafc53 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:02.831430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c55ad565-301a-48d8-8f69-8c4a83eafc53 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:02.831913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c55ad565-301a-48d8-8f69-8c4a83eafc53 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:02.832344 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c55ad565-301a-48d8-8f69-8c4a83eafc53 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:02.833037 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c55ad565-301a-48d8-8f69-8c4a83eafc53 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:02.833608 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c55ad565-301a-48d8-8f69-8c4a83eafc53 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:02.834035 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c55ad565-301a-48d8-8f69-8c4a83eafc53 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:02.865665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c55ad565-301a-48d8-8f69-8c4a83eafc53 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047488 Sep 08 14:01:02.866280 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 452/1012] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:02 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:03.413249 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-414c8f4c-0001-4d1e-9f51-aa2c07bb4fcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:03.415450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-414c8f4c-0001-4d1e-9f51-aa2c07bb4fcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:03.422462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-414c8f4c-0001-4d1e-9f51-aa2c07bb4fcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:03.422871 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-414c8f4c-0001-4d1e-9f51-aa2c07bb4fcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:03.423306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-414c8f4c-0001-4d1e-9f51-aa2c07bb4fcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:03.423990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-414c8f4c-0001-4d1e-9f51-aa2c07bb4fcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:03.424452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-414c8f4c-0001-4d1e-9f51-aa2c07bb4fcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:03.424855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-414c8f4c-0001-4d1e-9f51-aa2c07bb4fcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:03.455433 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-414c8f4c-0001-4d1e-9f51-aa2c07bb4fcc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046201 Sep 08 14:01:03.456028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 561/1013] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:03 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:03.879092 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-585433d7-fc9a-4dfb-b155-e669d76053f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:03.881219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-585433d7-fc9a-4dfb-b155-e669d76053f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:03.888456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-585433d7-fc9a-4dfb-b155-e669d76053f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:03.889061 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-585433d7-fc9a-4dfb-b155-e669d76053f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:03.889061 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-585433d7-fc9a-4dfb-b155-e669d76053f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:03.889655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-585433d7-fc9a-4dfb-b155-e669d76053f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:03.890104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-585433d7-fc9a-4dfb-b155-e669d76053f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:03.890554 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-585433d7-fc9a-4dfb-b155-e669d76053f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:03.922113 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-585433d7-fc9a-4dfb-b155-e669d76053f2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046943 Sep 08 14:01:03.922716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 453/1014] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:03 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:04.468147 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-dc8336a9-3840-42fc-af20-4ac6b6faae2d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:04.470449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-dc8336a9-3840-42fc-af20-4ac6b6faae2d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:04.477527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dc8336a9-3840-42fc-af20-4ac6b6faae2d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:04.478057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dc8336a9-3840-42fc-af20-4ac6b6faae2d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:04.478532 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dc8336a9-3840-42fc-af20-4ac6b6faae2d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:04.479393 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dc8336a9-3840-42fc-af20-4ac6b6faae2d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:04.479904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dc8336a9-3840-42fc-af20-4ac6b6faae2d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:04.480552 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dc8336a9-3840-42fc-af20-4ac6b6faae2d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:04.517516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-dc8336a9-3840-42fc-af20-4ac6b6faae2d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053382 Sep 08 14:01:04.518102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 562/1015] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:04 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:04.934899 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ef03dd13-e5df-4774-b5fb-a8487d19d5c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:04.937091 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ef03dd13-e5df-4774-b5fb-a8487d19d5c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:04.943355 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef03dd13-e5df-4774-b5fb-a8487d19d5c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:04.943842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef03dd13-e5df-4774-b5fb-a8487d19d5c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:04.944276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef03dd13-e5df-4774-b5fb-a8487d19d5c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:04.944954 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef03dd13-e5df-4774-b5fb-a8487d19d5c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:04.945391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef03dd13-e5df-4774-b5fb-a8487d19d5c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:04.945805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef03dd13-e5df-4774-b5fb-a8487d19d5c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:04.977297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ef03dd13-e5df-4774-b5fb-a8487d19d5c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046329 Sep 08 14:01:04.977878 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 454/1016] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:04 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:05.531135 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-613ad2b3-e7f3-411d-b575-19058f07262f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:05.533332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-613ad2b3-e7f3-411d-b575-19058f07262f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:05.539878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-613ad2b3-e7f3-411d-b575-19058f07262f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:05.540369 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-613ad2b3-e7f3-411d-b575-19058f07262f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:05.540817 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-613ad2b3-e7f3-411d-b575-19058f07262f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:05.541512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-613ad2b3-e7f3-411d-b575-19058f07262f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:05.542002 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-613ad2b3-e7f3-411d-b575-19058f07262f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:05.542477 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-613ad2b3-e7f3-411d-b575-19058f07262f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:05.573936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-613ad2b3-e7f3-411d-b575-19058f07262f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046735 Sep 08 14:01:05.574531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 563/1017] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:05 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:05.991302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4b9283ab-0fda-46ad-ace3-cab4ee348d87 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:05.993467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4b9283ab-0fda-46ad-ace3-cab4ee348d87 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:06.001086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b9283ab-0fda-46ad-ace3-cab4ee348d87 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:06.001718 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b9283ab-0fda-46ad-ace3-cab4ee348d87 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:06.002292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b9283ab-0fda-46ad-ace3-cab4ee348d87 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:06.003099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b9283ab-0fda-46ad-ace3-cab4ee348d87 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:06.003649 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b9283ab-0fda-46ad-ace3-cab4ee348d87 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:06.004164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b9283ab-0fda-46ad-ace3-cab4ee348d87 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:06.008501 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:06.048679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4b9283ab-0fda-46ad-ace3-cab4ee348d87 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061103 Sep 08 14:01:06.049649 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 455/1018] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:05 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:01:06.073152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-43e14d42-c434-4ca4-8946-7f6527f92915 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-656088964"}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:01:06.073688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-43e14d42-c434-4ca4-8946-7f6527f92915 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:06.081076 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43e14d42-c434-4ca4-8946-7f6527f92915 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:06.081823 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43e14d42-c434-4ca4-8946-7f6527f92915 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:06.082116 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43e14d42-c434-4ca4-8946-7f6527f92915 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:06.083676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43e14d42-c434-4ca4-8946-7f6527f92915 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:06.085021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43e14d42-c434-4ca4-8946-7f6527f92915 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:06.085021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43e14d42-c434-4ca4-8946-7f6527f92915 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:06.245279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.network.security_group_api [None req-43e14d42-c434-4ca4-8946-7f6527f92915 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Removing security group a479c883-ffb8-4546-b5d7-d2563c548f66 from port 48757fc0-370f-4fb9-a0d2-0688efed54e8 Sep 08 14:01:06.588125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2209af8c-ae87-4a8e-8885-5eb60d44f7ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:06.591164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2209af8c-ae87-4a8e-8885-5eb60d44f7ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:06.598684 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2209af8c-ae87-4a8e-8885-5eb60d44f7ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:06.599178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2209af8c-ae87-4a8e-8885-5eb60d44f7ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:06.599823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2209af8c-ae87-4a8e-8885-5eb60d44f7ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:06.600662 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2209af8c-ae87-4a8e-8885-5eb60d44f7ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:06.601119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2209af8c-ae87-4a8e-8885-5eb60d44f7ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:06.601542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2209af8c-ae87-4a8e-8885-5eb60d44f7ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:06.639429 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2209af8c-ae87-4a8e-8885-5eb60d44f7ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056127 Sep 08 14:01:06.641089 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 456/1019] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:06 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:06.717744 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-43e14d42-c434-4ca4-8946-7f6527f92915 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "POST /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/action" status: 202 len: 0 microversion: 2.1 time: 0.643370 Sep 08 14:01:06.717744 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 564/1020] 10.222.0.25 () {60 vars in 1278 bytes} [Thu Sep 8 14:01:06 2022] POST /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/action => generated 0 bytes in 645 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:01:06.724259 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-c1796928-031d-444d-9cad-33fdd5b89a26 req-a5565533-8226-4192-b9ba-1ee11b62acfe service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d7ae43e4-11c8-417c-be73-4f9fb5c42f2c", "tag": "48757fc0-370f-4fb9-a0d2-0688efed54e8"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:01:06.734742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-c1796928-031d-444d-9cad-33fdd5b89a26 req-a5565533-8226-4192-b9ba-1ee11b62acfe service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:06.736546 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-c1796928-031d-444d-9cad-33fdd5b89a26 req-a5565533-8226-4192-b9ba-1ee11b62acfe service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:06.737249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-c1796928-031d-444d-9cad-33fdd5b89a26 req-a5565533-8226-4192-b9ba-1ee11b62acfe service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:06.751288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-c1796928-031d-444d-9cad-33fdd5b89a26 req-a5565533-8226-4192-b9ba-1ee11b62acfe service nova] Creating event network-changed:48757fc0-370f-4fb9-a0d2-0688efed54e8 for instance d7ae43e4-11c8-417c-be73-4f9fb5c42f2c on hv-cinder-85646 Sep 08 14:01:06.757808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-c1796928-031d-444d-9cad-33fdd5b89a26 req-a5565533-8226-4192-b9ba-1ee11b62acfe service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037237 Sep 08 14:01:06.758998 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 457/1021] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:01:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:07.066741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-10fe6d00-ed52-4663-97b9-c969617fb991 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:07.069809 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-10fe6d00-ed52-4663-97b9-c969617fb991 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:07.077773 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-10fe6d00-ed52-4663-97b9-c969617fb991 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:07.078679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-10fe6d00-ed52-4663-97b9-c969617fb991 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:07.079713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-10fe6d00-ed52-4663-97b9-c969617fb991 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:07.080788 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-10fe6d00-ed52-4663-97b9-c969617fb991 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:07.081329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-10fe6d00-ed52-4663-97b9-c969617fb991 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:07.081838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-10fe6d00-ed52-4663-97b9-c969617fb991 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:07.129257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-10fe6d00-ed52-4663-97b9-c969617fb991 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.068066 Sep 08 14:01:07.130380 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 565/1022] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:07 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:07.654420 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f849964e-5710-48c9-afc7-8e73a7ebb011 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:07.656627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f849964e-5710-48c9-afc7-8e73a7ebb011 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:07.663385 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f849964e-5710-48c9-afc7-8e73a7ebb011 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:07.663859 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f849964e-5710-48c9-afc7-8e73a7ebb011 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:07.664287 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f849964e-5710-48c9-afc7-8e73a7ebb011 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:07.664972 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f849964e-5710-48c9-afc7-8e73a7ebb011 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:07.665528 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f849964e-5710-48c9-afc7-8e73a7ebb011 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:07.665942 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f849964e-5710-48c9-afc7-8e73a7ebb011 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:07.701977 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f849964e-5710-48c9-afc7-8e73a7ebb011 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052307 Sep 08 14:01:07.702478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 458/1023] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:07 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:08.147464 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1e0920e3-3159-4994-a00f-6acd7f581576 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:08.151059 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1e0920e3-3159-4994-a00f-6acd7f581576 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:08.162921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1e0920e3-3159-4994-a00f-6acd7f581576 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:08.163797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1e0920e3-3159-4994-a00f-6acd7f581576 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:08.164613 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1e0920e3-3159-4994-a00f-6acd7f581576 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:08.165863 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1e0920e3-3159-4994-a00f-6acd7f581576 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:08.166768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1e0920e3-3159-4994-a00f-6acd7f581576 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:08.167521 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1e0920e3-3159-4994-a00f-6acd7f581576 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:08.210713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1e0920e3-3159-4994-a00f-6acd7f581576 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.070564 Sep 08 14:01:08.211191 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 566/1024] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:08 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:08.471702 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-26b73dd3-d52d-4fb8-8801-8e495b7c2620 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:08.472184 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-26b73dd3-d52d-4fb8-8801-8e495b7c2620 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:08.478509 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26b73dd3-d52d-4fb8-8801-8e495b7c2620 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:08.478981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26b73dd3-d52d-4fb8-8801-8e495b7c2620 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:08.479584 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26b73dd3-d52d-4fb8-8801-8e495b7c2620 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:08.480267 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26b73dd3-d52d-4fb8-8801-8e495b7c2620 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:08.480719 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26b73dd3-d52d-4fb8-8801-8e495b7c2620 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:08.481133 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26b73dd3-d52d-4fb8-8801-8e495b7c2620 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:08.565123 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-26b73dd3-d52d-4fb8-8801-8e495b7c2620 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lazy-loading 'services' on Instance uuid d7ae43e4-11c8-417c-be73-4f9fb5c42f2c {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:01:08.714825 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d7d43133-23d5-4c44-bd14-236e5c4e2508 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:08.716926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d7d43133-23d5-4c44-bd14-236e5c4e2508 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:08.723021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d7d43133-23d5-4c44-bd14-236e5c4e2508 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:08.723511 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d7d43133-23d5-4c44-bd14-236e5c4e2508 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:08.723950 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d7d43133-23d5-4c44-bd14-236e5c4e2508 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:08.724857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d7d43133-23d5-4c44-bd14-236e5c4e2508 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:08.725311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d7d43133-23d5-4c44-bd14-236e5c4e2508 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:08.725737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d7d43133-23d5-4c44-bd14-236e5c4e2508 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:08.758345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d7d43133-23d5-4c44-bd14-236e5c4e2508 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047856 Sep 08 14:01:08.764308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 567/1025] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:08 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:08.775198 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-df6d4554-17fa-438e-af20-65f14201346a req-200a0a0d-a3e8-4bee-a8f8-43713ece7e8f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d7ae43e4-11c8-417c-be73-4f9fb5c42f2c", "tag": "48757fc0-370f-4fb9-a0d2-0688efed54e8"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:01:08.785259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-df6d4554-17fa-438e-af20-65f14201346a req-200a0a0d-a3e8-4bee-a8f8-43713ece7e8f service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:08.785873 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-df6d4554-17fa-438e-af20-65f14201346a req-200a0a0d-a3e8-4bee-a8f8-43713ece7e8f service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:08.786265 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-df6d4554-17fa-438e-af20-65f14201346a req-200a0a0d-a3e8-4bee-a8f8-43713ece7e8f service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:08.798357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-df6d4554-17fa-438e-af20-65f14201346a req-200a0a0d-a3e8-4bee-a8f8-43713ece7e8f service nova] Creating event network-changed:48757fc0-370f-4fb9-a0d2-0688efed54e8 for instance d7ae43e4-11c8-417c-be73-4f9fb5c42f2c on hv-cinder-85646 Sep 08 14:01:08.804018 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-df6d4554-17fa-438e-af20-65f14201346a req-200a0a0d-a3e8-4bee-a8f8-43713ece7e8f service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032650 Sep 08 14:01:08.804634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 568/1026] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:01:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:08.814062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-26b73dd3-d52d-4fb8-8801-8e495b7c2620 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65" status: 202 len: 0 microversion: 2.1 time: 0.346190 Sep 08 14:01:08.814959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 459/1027] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:01:08 2022] DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 => generated 0 bytes in 348 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:01:08.824196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5932961b-6019-41eb-94bb-5de1c4e04b1e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:08.826491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5932961b-6019-41eb-94bb-5de1c4e04b1e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:08.834223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5932961b-6019-41eb-94bb-5de1c4e04b1e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:08.834223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5932961b-6019-41eb-94bb-5de1c4e04b1e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:08.834828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5932961b-6019-41eb-94bb-5de1c4e04b1e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:08.835554 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5932961b-6019-41eb-94bb-5de1c4e04b1e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:08.836015 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5932961b-6019-41eb-94bb-5de1c4e04b1e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:08.836443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5932961b-6019-41eb-94bb-5de1c4e04b1e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:08.873104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5932961b-6019-41eb-94bb-5de1c4e04b1e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053789 Sep 08 14:01:08.876309 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 569/1028] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:08 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:08.894723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4adf4219-5e8f-4135-b42d-a120fbb62420 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:08.895375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4adf4219-5e8f-4135-b42d-a120fbb62420 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:08.903979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4adf4219-5e8f-4135-b42d-a120fbb62420 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:08.904889 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4adf4219-5e8f-4135-b42d-a120fbb62420 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:08.905431 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4adf4219-5e8f-4135-b42d-a120fbb62420 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:08.906426 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4adf4219-5e8f-4135-b42d-a120fbb62420 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:08.906874 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4adf4219-5e8f-4135-b42d-a120fbb62420 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:08.907491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4adf4219-5e8f-4135-b42d-a120fbb62420 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:08.933605 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4adf4219-5e8f-4135-b42d-a120fbb62420 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556/ips" status: 200 len: 148 microversion: 2.1 time: 0.043090 Sep 08 14:01:08.941612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 460/1029] 10.222.0.25 () {58 vars in 1248 bytes} [Thu Sep 8 14:01:08 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556/ips => generated 148 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:09.224107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-51de86a8-210e-46a0-a63b-f1b92e7e2565 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:09.227811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-51de86a8-210e-46a0-a63b-f1b92e7e2565 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:09.241038 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51de86a8-210e-46a0-a63b-f1b92e7e2565 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:09.241877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51de86a8-210e-46a0-a63b-f1b92e7e2565 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:09.242360 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51de86a8-210e-46a0-a63b-f1b92e7e2565 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:09.243072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51de86a8-210e-46a0-a63b-f1b92e7e2565 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:09.243520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51de86a8-210e-46a0-a63b-f1b92e7e2565 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:09.244224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51de86a8-210e-46a0-a63b-f1b92e7e2565 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:09.279142 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-51de86a8-210e-46a0-a63b-f1b92e7e2565 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.059757 Sep 08 14:01:09.280022 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 570/1030] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:09 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:09.773035 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-61cd219a-07bc-452e-b31c-8ade47057bbc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:09.775525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-61cd219a-07bc-452e-b31c-8ade47057bbc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:09.783133 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-61cd219a-07bc-452e-b31c-8ade47057bbc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:09.783650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-61cd219a-07bc-452e-b31c-8ade47057bbc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:09.784085 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-61cd219a-07bc-452e-b31c-8ade47057bbc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:09.784841 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-61cd219a-07bc-452e-b31c-8ade47057bbc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:09.785536 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-61cd219a-07bc-452e-b31c-8ade47057bbc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:09.786036 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-61cd219a-07bc-452e-b31c-8ade47057bbc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:09.830631 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-61cd219a-07bc-452e-b31c-8ade47057bbc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062044 Sep 08 14:01:09.831370 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 461/1031] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:09 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:09.886289 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2a0bae49-2ca7-40f2-9152-2c407d24915b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:09.889039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2a0bae49-2ca7-40f2-9152-2c407d24915b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:09.895046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a0bae49-2ca7-40f2-9152-2c407d24915b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:09.895538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a0bae49-2ca7-40f2-9152-2c407d24915b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:09.896015 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a0bae49-2ca7-40f2-9152-2c407d24915b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:09.896901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a0bae49-2ca7-40f2-9152-2c407d24915b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:09.897446 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a0bae49-2ca7-40f2-9152-2c407d24915b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:09.897873 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a0bae49-2ca7-40f2-9152-2c407d24915b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:09.931336 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2a0bae49-2ca7-40f2-9152-2c407d24915b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049219 Sep 08 14:01:09.931924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 571/1032] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:09 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:10.293442 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1e79010a-c3e0-44eb-b07c-a4392d50f471 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:10.295975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1e79010a-c3e0-44eb-b07c-a4392d50f471 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:10.302470 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1e79010a-c3e0-44eb-b07c-a4392d50f471 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:10.303020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1e79010a-c3e0-44eb-b07c-a4392d50f471 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:10.304577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1e79010a-c3e0-44eb-b07c-a4392d50f471 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:10.304577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1e79010a-c3e0-44eb-b07c-a4392d50f471 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:10.305040 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1e79010a-c3e0-44eb-b07c-a4392d50f471 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:10.305040 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1e79010a-c3e0-44eb-b07c-a4392d50f471 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:10.337522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1e79010a-c3e0-44eb-b07c-a4392d50f471 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048585 Sep 08 14:01:10.338207 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 462/1033] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:10 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:10.845834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1c8cd8f5-70b6-44d4-9194-3d5a7a314739 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:10.848255 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1c8cd8f5-70b6-44d4-9194-3d5a7a314739 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:10.856256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1c8cd8f5-70b6-44d4-9194-3d5a7a314739 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:10.856750 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1c8cd8f5-70b6-44d4-9194-3d5a7a314739 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:10.857189 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1c8cd8f5-70b6-44d4-9194-3d5a7a314739 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:10.857902 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1c8cd8f5-70b6-44d4-9194-3d5a7a314739 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:10.858392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1c8cd8f5-70b6-44d4-9194-3d5a7a314739 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:10.858824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1c8cd8f5-70b6-44d4-9194-3d5a7a314739 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:10.893066 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1c8cd8f5-70b6-44d4-9194-3d5a7a314739 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051269 Sep 08 14:01:10.893639 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 572/1034] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:10 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:10.944982 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3489d326-0c77-45d4-bb44-e4504c30d3c7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:10.947246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3489d326-0c77-45d4-bb44-e4504c30d3c7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:10.954262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3489d326-0c77-45d4-bb44-e4504c30d3c7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:10.954749 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3489d326-0c77-45d4-bb44-e4504c30d3c7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:10.955187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3489d326-0c77-45d4-bb44-e4504c30d3c7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:10.955919 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3489d326-0c77-45d4-bb44-e4504c30d3c7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:10.956365 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3489d326-0c77-45d4-bb44-e4504c30d3c7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:10.956785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3489d326-0c77-45d4-bb44-e4504c30d3c7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:10.990327 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3489d326-0c77-45d4-bb44-e4504c30d3c7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049402 Sep 08 14:01:10.990828 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 463/1035] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:10 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:11.351239 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-133493d7-d72b-4657-833e-159c3e438ee4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:11.353428 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-133493d7-d72b-4657-833e-159c3e438ee4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:11.359600 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-133493d7-d72b-4657-833e-159c3e438ee4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:11.360066 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-133493d7-d72b-4657-833e-159c3e438ee4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:11.360511 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-133493d7-d72b-4657-833e-159c3e438ee4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:11.361215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-133493d7-d72b-4657-833e-159c3e438ee4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:11.361695 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-133493d7-d72b-4657-833e-159c3e438ee4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:11.362112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-133493d7-d72b-4657-833e-159c3e438ee4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:11.396824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-133493d7-d72b-4657-833e-159c3e438ee4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049151 Sep 08 14:01:11.397814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 573/1036] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:11 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:11.906396 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-45bd1ba2-71e1-4f54-a547-20bc88872da2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:11.908485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-45bd1ba2-71e1-4f54-a547-20bc88872da2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:11.915061 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-45bd1ba2-71e1-4f54-a547-20bc88872da2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:11.915547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-45bd1ba2-71e1-4f54-a547-20bc88872da2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:11.916047 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-45bd1ba2-71e1-4f54-a547-20bc88872da2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:11.916747 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-45bd1ba2-71e1-4f54-a547-20bc88872da2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:11.917202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-45bd1ba2-71e1-4f54-a547-20bc88872da2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:11.918161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-45bd1ba2-71e1-4f54-a547-20bc88872da2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:11.950895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-45bd1ba2-71e1-4f54-a547-20bc88872da2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048698 Sep 08 14:01:11.951547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 464/1037] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:11 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:12.003256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3d964b4c-e328-441b-84f5-78d7465c78e0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:12.005395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3d964b4c-e328-441b-84f5-78d7465c78e0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:12.012313 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d964b4c-e328-441b-84f5-78d7465c78e0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:12.012800 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d964b4c-e328-441b-84f5-78d7465c78e0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:12.013235 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d964b4c-e328-441b-84f5-78d7465c78e0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:12.013930 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d964b4c-e328-441b-84f5-78d7465c78e0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:12.014417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d964b4c-e328-441b-84f5-78d7465c78e0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:12.014866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d964b4c-e328-441b-84f5-78d7465c78e0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:12.044890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3d964b4c-e328-441b-84f5-78d7465c78e0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045595 Sep 08 14:01:12.045476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 574/1038] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:11 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:12.414229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b27dcff5-f906-4ee1-91c6-b95515bbaf2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:12.417608 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b27dcff5-f906-4ee1-91c6-b95515bbaf2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:12.428434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b27dcff5-f906-4ee1-91c6-b95515bbaf2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:12.429148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b27dcff5-f906-4ee1-91c6-b95515bbaf2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:12.429592 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b27dcff5-f906-4ee1-91c6-b95515bbaf2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:12.430539 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b27dcff5-f906-4ee1-91c6-b95515bbaf2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:12.431037 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b27dcff5-f906-4ee1-91c6-b95515bbaf2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:12.431506 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b27dcff5-f906-4ee1-91c6-b95515bbaf2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:12.463570 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b27dcff5-f906-4ee1-91c6-b95515bbaf2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053527 Sep 08 14:01:12.464159 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 465/1039] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:12 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:01:12.964045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-df5909ba-ccea-4311-8c59-656b5970446f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:12.966222 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-df5909ba-ccea-4311-8c59-656b5970446f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:12.972965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df5909ba-ccea-4311-8c59-656b5970446f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:12.973461 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df5909ba-ccea-4311-8c59-656b5970446f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:12.973934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df5909ba-ccea-4311-8c59-656b5970446f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:12.974687 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df5909ba-ccea-4311-8c59-656b5970446f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:12.975148 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df5909ba-ccea-4311-8c59-656b5970446f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:12.975580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df5909ba-ccea-4311-8c59-656b5970446f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:13.017132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-df5909ba-ccea-4311-8c59-656b5970446f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057399 Sep 08 14:01:13.017882 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 575/1040] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:12 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:13.057714 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fb8cc01b-9d8d-4fd6-8257-ee973e071dde tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:13.060429 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fb8cc01b-9d8d-4fd6-8257-ee973e071dde tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:13.062549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=68 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:13.067701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb8cc01b-9d8d-4fd6-8257-ee973e071dde tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:13.068187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb8cc01b-9d8d-4fd6-8257-ee973e071dde tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:13.068609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb8cc01b-9d8d-4fd6-8257-ee973e071dde tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:13.069311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb8cc01b-9d8d-4fd6-8257-ee973e071dde tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:13.069887 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb8cc01b-9d8d-4fd6-8257-ee973e071dde tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:13.070432 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb8cc01b-9d8d-4fd6-8257-ee973e071dde tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:13.110178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fb8cc01b-9d8d-4fd6-8257-ee973e071dde tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056636 Sep 08 14:01:13.110758 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 466/1041] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:13 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:13.477468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9f851511-36dc-46c8-a966-e7a32627fac2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:13.480219 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9f851511-36dc-46c8-a966-e7a32627fac2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:13.486531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9f851511-36dc-46c8-a966-e7a32627fac2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:13.487060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9f851511-36dc-46c8-a966-e7a32627fac2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:13.487633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9f851511-36dc-46c8-a966-e7a32627fac2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:13.488418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9f851511-36dc-46c8-a966-e7a32627fac2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:13.489025 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9f851511-36dc-46c8-a966-e7a32627fac2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:13.490259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9f851511-36dc-46c8-a966-e7a32627fac2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:13.535252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9f851511-36dc-46c8-a966-e7a32627fac2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.062057 Sep 08 14:01:13.535897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 576/1042] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:13 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:14.031824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7c423921-df7f-4b38-b37b-4ac98622b9e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:14.034614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7c423921-df7f-4b38-b37b-4ac98622b9e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:14.043411 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c423921-df7f-4b38-b37b-4ac98622b9e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:14.043901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c423921-df7f-4b38-b37b-4ac98622b9e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:14.044396 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c423921-df7f-4b38-b37b-4ac98622b9e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:14.045107 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c423921-df7f-4b38-b37b-4ac98622b9e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:14.045599 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c423921-df7f-4b38-b37b-4ac98622b9e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:14.046028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c423921-df7f-4b38-b37b-4ac98622b9e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:14.082702 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7c423921-df7f-4b38-b37b-4ac98622b9e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056786 Sep 08 14:01:14.083430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 467/1043] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:14 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:14.123405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d05c6fc5-2cb6-4288-904d-858c341a0c81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:14.125644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d05c6fc5-2cb6-4288-904d-858c341a0c81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:14.132414 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d05c6fc5-2cb6-4288-904d-858c341a0c81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:14.132900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d05c6fc5-2cb6-4288-904d-858c341a0c81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:14.133601 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d05c6fc5-2cb6-4288-904d-858c341a0c81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:14.134534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d05c6fc5-2cb6-4288-904d-858c341a0c81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:14.135060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d05c6fc5-2cb6-4288-904d-858c341a0c81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:14.135555 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d05c6fc5-2cb6-4288-904d-858c341a0c81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:14.152063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:14.170171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d05c6fc5-2cb6-4288-904d-858c341a0c81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050974 Sep 08 14:01:14.170685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 577/1044] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:14 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:14.549878 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-13e57083-3568-42ca-b458-53a1f0dd01f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:14.552644 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-13e57083-3568-42ca-b458-53a1f0dd01f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:14.559382 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-13e57083-3568-42ca-b458-53a1f0dd01f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:14.559862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-13e57083-3568-42ca-b458-53a1f0dd01f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:14.560293 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-13e57083-3568-42ca-b458-53a1f0dd01f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:14.560984 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-13e57083-3568-42ca-b458-53a1f0dd01f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:14.561426 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-13e57083-3568-42ca-b458-53a1f0dd01f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:14.561847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-13e57083-3568-42ca-b458-53a1f0dd01f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:14.601119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-13e57083-3568-42ca-b458-53a1f0dd01f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055730 Sep 08 14:01:14.601713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 468/1045] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:14 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:01:15.096165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8bcefe75-00ab-4df0-9b8b-c58227c4e3e0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:15.098606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8bcefe75-00ab-4df0-9b8b-c58227c4e3e0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:15.105778 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8bcefe75-00ab-4df0-9b8b-c58227c4e3e0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:15.106452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8bcefe75-00ab-4df0-9b8b-c58227c4e3e0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:15.107010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8bcefe75-00ab-4df0-9b8b-c58227c4e3e0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:15.108100 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8bcefe75-00ab-4df0-9b8b-c58227c4e3e0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:15.108644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8bcefe75-00ab-4df0-9b8b-c58227c4e3e0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:15.109121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8bcefe75-00ab-4df0-9b8b-c58227c4e3e0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:15.141832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8bcefe75-00ab-4df0-9b8b-c58227c4e3e0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049567 Sep 08 14:01:15.142655 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 578/1046] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:15 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:15.183994 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-18d84d46-f385-4d70-a636-8a93e53d23fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:15.186056 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-18d84d46-f385-4d70-a636-8a93e53d23fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:15.192815 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18d84d46-f385-4d70-a636-8a93e53d23fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:15.193295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18d84d46-f385-4d70-a636-8a93e53d23fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:15.202413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18d84d46-f385-4d70-a636-8a93e53d23fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:15.202413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18d84d46-f385-4d70-a636-8a93e53d23fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:15.202413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18d84d46-f385-4d70-a636-8a93e53d23fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:15.202413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18d84d46-f385-4d70-a636-8a93e53d23fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:15.231956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-18d84d46-f385-4d70-a636-8a93e53d23fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051929 Sep 08 14:01:15.232559 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 469/1047] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:15 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:15.618612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ae957b39-f47a-40cf-a933-7fc6588e4eb2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:15.620791 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ae957b39-f47a-40cf-a933-7fc6588e4eb2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:15.627135 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae957b39-f47a-40cf-a933-7fc6588e4eb2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:15.627689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae957b39-f47a-40cf-a933-7fc6588e4eb2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:15.628216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae957b39-f47a-40cf-a933-7fc6588e4eb2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:15.628895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae957b39-f47a-40cf-a933-7fc6588e4eb2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:15.629468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae957b39-f47a-40cf-a933-7fc6588e4eb2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:15.629824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae957b39-f47a-40cf-a933-7fc6588e4eb2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:15.668628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ae957b39-f47a-40cf-a933-7fc6588e4eb2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.056656 Sep 08 14:01:15.669132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 579/1048] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:15 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:16.156101 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0e7b27ee-4be1-46a8-b92e-54df2794cbb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:16.158647 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0e7b27ee-4be1-46a8-b92e-54df2794cbb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:16.165918 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e7b27ee-4be1-46a8-b92e-54df2794cbb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:16.166500 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e7b27ee-4be1-46a8-b92e-54df2794cbb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:16.167017 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e7b27ee-4be1-46a8-b92e-54df2794cbb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:16.168017 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e7b27ee-4be1-46a8-b92e-54df2794cbb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:16.168788 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e7b27ee-4be1-46a8-b92e-54df2794cbb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:16.169320 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e7b27ee-4be1-46a8-b92e-54df2794cbb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:16.203852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0e7b27ee-4be1-46a8-b92e-54df2794cbb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052023 Sep 08 14:01:16.204893 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 470/1049] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:16 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:16.246244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-af8b4a80-9e76-4766-a1d1-cfcfdcd71e54 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:16.248336 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-af8b4a80-9e76-4766-a1d1-cfcfdcd71e54 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:16.255750 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af8b4a80-9e76-4766-a1d1-cfcfdcd71e54 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:16.256338 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af8b4a80-9e76-4766-a1d1-cfcfdcd71e54 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:16.256819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af8b4a80-9e76-4766-a1d1-cfcfdcd71e54 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:16.257573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af8b4a80-9e76-4766-a1d1-cfcfdcd71e54 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:16.258078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af8b4a80-9e76-4766-a1d1-cfcfdcd71e54 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:16.258605 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af8b4a80-9e76-4766-a1d1-cfcfdcd71e54 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:16.296687 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-af8b4a80-9e76-4766-a1d1-cfcfdcd71e54 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055408 Sep 08 14:01:16.297431 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 580/1050] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:16 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:16.684575 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-31959500-ecc9-4fe9-affb-bfcd5f5e50b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:16.686856 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-31959500-ecc9-4fe9-affb-bfcd5f5e50b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:16.693547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31959500-ecc9-4fe9-affb-bfcd5f5e50b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:16.694304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31959500-ecc9-4fe9-affb-bfcd5f5e50b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:16.694931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31959500-ecc9-4fe9-affb-bfcd5f5e50b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:16.695826 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31959500-ecc9-4fe9-affb-bfcd5f5e50b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:16.696286 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31959500-ecc9-4fe9-affb-bfcd5f5e50b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:16.696717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31959500-ecc9-4fe9-affb-bfcd5f5e50b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:16.739162 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-31959500-ecc9-4fe9-affb-bfcd5f5e50b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058793 Sep 08 14:01:16.739991 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 471/1051] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:16 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:01:17.221125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-dd2f5c36-d61a-4a86-88d1-715d8223dd13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:17.225062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-dd2f5c36-d61a-4a86-88d1-715d8223dd13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:17.235449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dd2f5c36-d61a-4a86-88d1-715d8223dd13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:17.235959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dd2f5c36-d61a-4a86-88d1-715d8223dd13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:17.236404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dd2f5c36-d61a-4a86-88d1-715d8223dd13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:17.237127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dd2f5c36-d61a-4a86-88d1-715d8223dd13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:17.237584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dd2f5c36-d61a-4a86-88d1-715d8223dd13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:17.238013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dd2f5c36-d61a-4a86-88d1-715d8223dd13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:17.281305 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-dd2f5c36-d61a-4a86-88d1-715d8223dd13 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067498 Sep 08 14:01:17.282367 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 581/1052] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:17 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:17.311903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1b0d306d-4216-4c21-b49c-9c07491a5b3c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:17.313991 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1b0d306d-4216-4c21-b49c-9c07491a5b3c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:17.321918 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b0d306d-4216-4c21-b49c-9c07491a5b3c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:17.324453 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b0d306d-4216-4c21-b49c-9c07491a5b3c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:17.324453 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b0d306d-4216-4c21-b49c-9c07491a5b3c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:17.324742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b0d306d-4216-4c21-b49c-9c07491a5b3c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:17.325352 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b0d306d-4216-4c21-b49c-9c07491a5b3c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:17.325852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b0d306d-4216-4c21-b49c-9c07491a5b3c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:17.330076 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:17.382462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1b0d306d-4216-4c21-b49c-9c07491a5b3c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.075740 Sep 08 14:01:17.383535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 472/1053] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:17 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:17.756076 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-246b81eb-9cda-4ae3-a744-fab96b98dd15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:17.758456 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-246b81eb-9cda-4ae3-a744-fab96b98dd15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:17.765831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-246b81eb-9cda-4ae3-a744-fab96b98dd15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:17.766366 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-246b81eb-9cda-4ae3-a744-fab96b98dd15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:17.766807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-246b81eb-9cda-4ae3-a744-fab96b98dd15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:17.767514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-246b81eb-9cda-4ae3-a744-fab96b98dd15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:17.767970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-246b81eb-9cda-4ae3-a744-fab96b98dd15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:17.768398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-246b81eb-9cda-4ae3-a744-fab96b98dd15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:17.806054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-246b81eb-9cda-4ae3-a744-fab96b98dd15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054235 Sep 08 14:01:17.806667 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 582/1054] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:17 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:18.295953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7c77a2db-4f51-4a46-9563-9e4128f6cb66 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:18.298524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7c77a2db-4f51-4a46-9563-9e4128f6cb66 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:18.305941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c77a2db-4f51-4a46-9563-9e4128f6cb66 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:18.306920 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c77a2db-4f51-4a46-9563-9e4128f6cb66 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:18.307622 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c77a2db-4f51-4a46-9563-9e4128f6cb66 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:18.308781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c77a2db-4f51-4a46-9563-9e4128f6cb66 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:18.309444 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c77a2db-4f51-4a46-9563-9e4128f6cb66 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:18.309879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c77a2db-4f51-4a46-9563-9e4128f6cb66 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:18.348646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7c77a2db-4f51-4a46-9563-9e4128f6cb66 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057528 Sep 08 14:01:18.349478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 473/1055] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:18 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:18.401493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-eb6bcaec-8a5e-4f77-a236-110c0293d8cd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:18.403989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-eb6bcaec-8a5e-4f77-a236-110c0293d8cd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:18.410917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb6bcaec-8a5e-4f77-a236-110c0293d8cd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:18.411288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb6bcaec-8a5e-4f77-a236-110c0293d8cd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:18.411733 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb6bcaec-8a5e-4f77-a236-110c0293d8cd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:18.412444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb6bcaec-8a5e-4f77-a236-110c0293d8cd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:18.412896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb6bcaec-8a5e-4f77-a236-110c0293d8cd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:18.413387 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb6bcaec-8a5e-4f77-a236-110c0293d8cd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:18.451519 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-eb6bcaec-8a5e-4f77-a236-110c0293d8cd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054776 Sep 08 14:01:18.452224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 583/1056] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:18 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:18.824540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d2a86057-d611-42fa-be27-98c2901f3159 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:18.828007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d2a86057-d611-42fa-be27-98c2901f3159 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:18.837813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d2a86057-d611-42fa-be27-98c2901f3159 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:18.838373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d2a86057-d611-42fa-be27-98c2901f3159 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:18.838823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d2a86057-d611-42fa-be27-98c2901f3159 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:18.839987 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d2a86057-d611-42fa-be27-98c2901f3159 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:18.840797 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d2a86057-d611-42fa-be27-98c2901f3159 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:18.841571 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d2a86057-d611-42fa-be27-98c2901f3159 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:18.893418 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d2a86057-d611-42fa-be27-98c2901f3159 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.076651 Sep 08 14:01:18.894102 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 474/1057] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:18 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:19.362854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-27c41014-77f9-44df-b211-4e7f921de5b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:19.366026 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-27c41014-77f9-44df-b211-4e7f921de5b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:19.373831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27c41014-77f9-44df-b211-4e7f921de5b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:19.374427 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27c41014-77f9-44df-b211-4e7f921de5b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:19.375009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27c41014-77f9-44df-b211-4e7f921de5b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:19.375758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27c41014-77f9-44df-b211-4e7f921de5b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:19.376311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27c41014-77f9-44df-b211-4e7f921de5b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:19.376794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27c41014-77f9-44df-b211-4e7f921de5b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:19.417651 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-27c41014-77f9-44df-b211-4e7f921de5b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059806 Sep 08 14:01:19.418600 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 584/1058] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:19 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:19.470256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-69a51f9b-5145-454d-b287-ad65467dbb33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:19.473955 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-69a51f9b-5145-454d-b287-ad65467dbb33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:19.486100 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-69a51f9b-5145-454d-b287-ad65467dbb33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:19.487023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-69a51f9b-5145-454d-b287-ad65467dbb33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:19.487799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-69a51f9b-5145-454d-b287-ad65467dbb33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:19.489030 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-69a51f9b-5145-454d-b287-ad65467dbb33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:19.490058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-69a51f9b-5145-454d-b287-ad65467dbb33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:19.490890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-69a51f9b-5145-454d-b287-ad65467dbb33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:19.535945 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-69a51f9b-5145-454d-b287-ad65467dbb33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.074088 Sep 08 14:01:19.536595 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 475/1059] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:19 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:19.912367 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-09f302c4-aa91-48f9-a322-c3580674591e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:19.916169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-09f302c4-aa91-48f9-a322-c3580674591e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:19.926534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-09f302c4-aa91-48f9-a322-c3580674591e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:19.927118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-09f302c4-aa91-48f9-a322-c3580674591e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:19.927773 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-09f302c4-aa91-48f9-a322-c3580674591e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:19.928996 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-09f302c4-aa91-48f9-a322-c3580674591e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:19.929689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-09f302c4-aa91-48f9-a322-c3580674591e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:19.930366 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-09f302c4-aa91-48f9-a322-c3580674591e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:19.984888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-09f302c4-aa91-48f9-a322-c3580674591e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.079592 Sep 08 14:01:19.985775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 585/1060] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:19 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:20.433559 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c1d7ee76-a2ce-4290-9153-82d099ee43fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:20.436409 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c1d7ee76-a2ce-4290-9153-82d099ee43fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:20.445494 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c1d7ee76-a2ce-4290-9153-82d099ee43fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:20.446634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c1d7ee76-a2ce-4290-9153-82d099ee43fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:20.448015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c1d7ee76-a2ce-4290-9153-82d099ee43fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:20.449323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c1d7ee76-a2ce-4290-9153-82d099ee43fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:20.450169 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c1d7ee76-a2ce-4290-9153-82d099ee43fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:20.450956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c1d7ee76-a2ce-4290-9153-82d099ee43fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:20.512400 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c1d7ee76-a2ce-4290-9153-82d099ee43fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.083243 Sep 08 14:01:20.513425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 476/1061] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:20 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:20.551301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-276732b6-33c8-49a9-a533-6f5f4b1e245d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:20.553371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-276732b6-33c8-49a9-a533-6f5f4b1e245d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:20.561257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-276732b6-33c8-49a9-a533-6f5f4b1e245d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:20.561749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-276732b6-33c8-49a9-a533-6f5f4b1e245d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:20.562199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-276732b6-33c8-49a9-a533-6f5f4b1e245d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:20.562901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-276732b6-33c8-49a9-a533-6f5f4b1e245d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:20.563350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-276732b6-33c8-49a9-a533-6f5f4b1e245d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:20.563764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-276732b6-33c8-49a9-a533-6f5f4b1e245d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:20.599912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-276732b6-33c8-49a9-a533-6f5f4b1e245d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052677 Sep 08 14:01:20.600706 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 586/1062] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:20 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:21.005492 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bda49df0-a165-4454-a30d-18cdb627cd4b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:21.008959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bda49df0-a165-4454-a30d-18cdb627cd4b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:21.019469 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bda49df0-a165-4454-a30d-18cdb627cd4b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:21.020317 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bda49df0-a165-4454-a30d-18cdb627cd4b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:21.021089 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bda49df0-a165-4454-a30d-18cdb627cd4b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:21.022369 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bda49df0-a165-4454-a30d-18cdb627cd4b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:21.023120 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bda49df0-a165-4454-a30d-18cdb627cd4b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:21.023758 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bda49df0-a165-4454-a30d-18cdb627cd4b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:21.083758 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bda49df0-a165-4454-a30d-18cdb627cd4b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.085135 Sep 08 14:01:21.084831 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 477/1063] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:20 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:21.532236 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-69311322-dcab-4297-b27a-3dfaebde3ff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:21.534862 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-69311322-dcab-4297-b27a-3dfaebde3ff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:21.545680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69311322-dcab-4297-b27a-3dfaebde3ff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:21.546526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69311322-dcab-4297-b27a-3dfaebde3ff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:21.547227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69311322-dcab-4297-b27a-3dfaebde3ff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:21.548341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69311322-dcab-4297-b27a-3dfaebde3ff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:21.549103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69311322-dcab-4297-b27a-3dfaebde3ff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:21.549918 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69311322-dcab-4297-b27a-3dfaebde3ff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:21.606220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-69311322-dcab-4297-b27a-3dfaebde3ff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.080612 Sep 08 14:01:21.607428 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 587/1064] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:21 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:21.616060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f8dcc1d3-1cc4-4ab0-acf6-8996a41e07d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:21.618225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f8dcc1d3-1cc4-4ab0-acf6-8996a41e07d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:21.625686 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f8dcc1d3-1cc4-4ab0-acf6-8996a41e07d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:21.626204 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f8dcc1d3-1cc4-4ab0-acf6-8996a41e07d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:21.628895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f8dcc1d3-1cc4-4ab0-acf6-8996a41e07d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:21.628895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f8dcc1d3-1cc4-4ab0-acf6-8996a41e07d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:21.628895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f8dcc1d3-1cc4-4ab0-acf6-8996a41e07d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:21.628895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f8dcc1d3-1cc4-4ab0-acf6-8996a41e07d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:21.667783 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f8dcc1d3-1cc4-4ab0-acf6-8996a41e07d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057444 Sep 08 14:01:21.668433 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 478/1065] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:21 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:22.101657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-afc673fe-f4ca-4402-a173-8a8dc5b2d3cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:22.104153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-afc673fe-f4ca-4402-a173-8a8dc5b2d3cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:22.110714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-afc673fe-f4ca-4402-a173-8a8dc5b2d3cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:22.111355 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-afc673fe-f4ca-4402-a173-8a8dc5b2d3cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:22.111951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-afc673fe-f4ca-4402-a173-8a8dc5b2d3cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:22.112746 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-afc673fe-f4ca-4402-a173-8a8dc5b2d3cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:22.113209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-afc673fe-f4ca-4402-a173-8a8dc5b2d3cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:22.113755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-afc673fe-f4ca-4402-a173-8a8dc5b2d3cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:22.148295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-afc673fe-f4ca-4402-a173-8a8dc5b2d3cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051476 Sep 08 14:01:22.148993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 588/1066] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:22 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:01:22.623049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4b8926cb-dbbd-4e1e-bfad-3824c1fe9890 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:22.625336 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4b8926cb-dbbd-4e1e-bfad-3824c1fe9890 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:22.635391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b8926cb-dbbd-4e1e-bfad-3824c1fe9890 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:22.636164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b8926cb-dbbd-4e1e-bfad-3824c1fe9890 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:22.636981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b8926cb-dbbd-4e1e-bfad-3824c1fe9890 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:22.638114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b8926cb-dbbd-4e1e-bfad-3824c1fe9890 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:22.638882 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b8926cb-dbbd-4e1e-bfad-3824c1fe9890 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:22.639536 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b8926cb-dbbd-4e1e-bfad-3824c1fe9890 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:22.678615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4b8926cb-dbbd-4e1e-bfad-3824c1fe9890 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059882 Sep 08 14:01:22.679183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 479/1067] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:22 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:22.681469 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-993c28f7-faa8-4cbf-80b5-ce842d978ac7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:22.683840 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-993c28f7-faa8-4cbf-80b5-ce842d978ac7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:22.695900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-993c28f7-faa8-4cbf-80b5-ce842d978ac7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:22.696402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-993c28f7-faa8-4cbf-80b5-ce842d978ac7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:22.697404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-993c28f7-faa8-4cbf-80b5-ce842d978ac7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:22.698107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-993c28f7-faa8-4cbf-80b5-ce842d978ac7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:22.698590 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-993c28f7-faa8-4cbf-80b5-ce842d978ac7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:22.699018 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-993c28f7-faa8-4cbf-80b5-ce842d978ac7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:22.732355 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-993c28f7-faa8-4cbf-80b5-ce842d978ac7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055279 Sep 08 14:01:22.732949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 589/1068] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:22 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:23.163587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-08eb3e75-1fc0-4cd1-b8c2-16b00d351bfd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:23.166896 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-08eb3e75-1fc0-4cd1-b8c2-16b00d351bfd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:23.175636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08eb3e75-1fc0-4cd1-b8c2-16b00d351bfd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:23.176129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08eb3e75-1fc0-4cd1-b8c2-16b00d351bfd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:23.177146 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08eb3e75-1fc0-4cd1-b8c2-16b00d351bfd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:23.178268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08eb3e75-1fc0-4cd1-b8c2-16b00d351bfd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:23.179453 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08eb3e75-1fc0-4cd1-b8c2-16b00d351bfd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:23.180610 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-08eb3e75-1fc0-4cd1-b8c2-16b00d351bfd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:23.220077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-08eb3e75-1fc0-4cd1-b8c2-16b00d351bfd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061407 Sep 08 14:01:23.220655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 480/1069] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:23 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:23.692364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fbd63296-e319-4037-86aa-0d53c11d72a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:23.694613 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fbd63296-e319-4037-86aa-0d53c11d72a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:23.701654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fbd63296-e319-4037-86aa-0d53c11d72a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:23.702305 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fbd63296-e319-4037-86aa-0d53c11d72a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:23.702801 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fbd63296-e319-4037-86aa-0d53c11d72a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:23.703566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fbd63296-e319-4037-86aa-0d53c11d72a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:23.704072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fbd63296-e319-4037-86aa-0d53c11d72a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:23.704551 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fbd63296-e319-4037-86aa-0d53c11d72a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:23.748055 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-627777ce-586b-49f5-b5a8-624e50e09aaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:23.751821 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-627777ce-586b-49f5-b5a8-624e50e09aaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:23.756053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fbd63296-e319-4037-86aa-0d53c11d72a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067529 Sep 08 14:01:23.757110 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 590/1070] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:23 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:23.761930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-627777ce-586b-49f5-b5a8-624e50e09aaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:23.762823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-627777ce-586b-49f5-b5a8-624e50e09aaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:23.763611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-627777ce-586b-49f5-b5a8-624e50e09aaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:23.764719 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-627777ce-586b-49f5-b5a8-624e50e09aaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:23.765456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-627777ce-586b-49f5-b5a8-624e50e09aaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:23.766181 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-627777ce-586b-49f5-b5a8-624e50e09aaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:23.821360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-627777ce-586b-49f5-b5a8-624e50e09aaa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.078195 Sep 08 14:01:23.821360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 481/1071] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:23 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:24.234474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b4cfa1d7-c1ca-46c3-894a-92b270d75ea1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:24.236712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b4cfa1d7-c1ca-46c3-894a-92b270d75ea1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:24.244696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b4cfa1d7-c1ca-46c3-894a-92b270d75ea1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:24.245184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b4cfa1d7-c1ca-46c3-894a-92b270d75ea1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:24.245621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b4cfa1d7-c1ca-46c3-894a-92b270d75ea1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:24.246379 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b4cfa1d7-c1ca-46c3-894a-92b270d75ea1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:24.246844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b4cfa1d7-c1ca-46c3-894a-92b270d75ea1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:24.247276 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b4cfa1d7-c1ca-46c3-894a-92b270d75ea1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:24.291640 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b4cfa1d7-c1ca-46c3-894a-92b270d75ea1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061412 Sep 08 14:01:24.292245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 591/1072] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:24 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:24.771822 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a77b190d-c97f-4e9b-a930-ee2641390fe5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:24.775278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a77b190d-c97f-4e9b-a930-ee2641390fe5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:24.782067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a77b190d-c97f-4e9b-a930-ee2641390fe5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:24.782647 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a77b190d-c97f-4e9b-a930-ee2641390fe5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:24.783096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a77b190d-c97f-4e9b-a930-ee2641390fe5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:24.783812 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a77b190d-c97f-4e9b-a930-ee2641390fe5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:24.784260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a77b190d-c97f-4e9b-a930-ee2641390fe5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:24.784686 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a77b190d-c97f-4e9b-a930-ee2641390fe5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:24.818703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a77b190d-c97f-4e9b-a930-ee2641390fe5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051316 Sep 08 14:01:24.819483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 482/1073] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:24 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:24.839979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ec4dfee7-171f-4eba-8902-c1774411108a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:24.842500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ec4dfee7-171f-4eba-8902-c1774411108a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:24.851145 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec4dfee7-171f-4eba-8902-c1774411108a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:24.851942 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec4dfee7-171f-4eba-8902-c1774411108a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:24.853179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec4dfee7-171f-4eba-8902-c1774411108a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:24.854487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec4dfee7-171f-4eba-8902-c1774411108a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:24.855216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec4dfee7-171f-4eba-8902-c1774411108a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:24.855956 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec4dfee7-171f-4eba-8902-c1774411108a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:24.898752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ec4dfee7-171f-4eba-8902-c1774411108a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062902 Sep 08 14:01:24.899303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 592/1074] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:24 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:25.309292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1ad78ad6-2030-429e-95e8-3261d0c64364 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:25.313641 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1ad78ad6-2030-429e-95e8-3261d0c64364 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:25.322784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1ad78ad6-2030-429e-95e8-3261d0c64364 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:25.324894 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1ad78ad6-2030-429e-95e8-3261d0c64364 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:25.324894 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1ad78ad6-2030-429e-95e8-3261d0c64364 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:25.324894 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1ad78ad6-2030-429e-95e8-3261d0c64364 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:25.325394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1ad78ad6-2030-429e-95e8-3261d0c64364 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:25.325786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1ad78ad6-2030-429e-95e8-3261d0c64364 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:25.362661 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1ad78ad6-2030-429e-95e8-3261d0c64364 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.060874 Sep 08 14:01:25.363596 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 483/1075] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:25 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:25.580231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3fb768c8-4f2b-4e00-ab12-c91d18817282 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:01:25.582547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3fb768c8-4f2b-4e00-ab12-c91d18817282 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:25.584490 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=82 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:25.590903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3fb768c8-4f2b-4e00-ab12-c91d18817282 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:25.593710 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3fb768c8-4f2b-4e00-ab12-c91d18817282 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:25.593710 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3fb768c8-4f2b-4e00-ab12-c91d18817282 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:25.593710 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3fb768c8-4f2b-4e00-ab12-c91d18817282 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:25.593710 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3fb768c8-4f2b-4e00-ab12-c91d18817282 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:25.594232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3fb768c8-4f2b-4e00-ab12-c91d18817282 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:25.705369 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3fb768c8-4f2b-4e00-ab12-c91d18817282 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "POST /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94/action" status: 200 len: 14 microversion: 2.1 time: 0.128990 Sep 08 14:01:25.706128 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 593/1076] 10.222.0.25 () {60 vars in 1278 bytes} [Thu Sep 8 14:01:25 2022] POST /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94/action => generated 14 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 08 14:01:25.836057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-32455274-0966-4e06-ab75-294dd597c86f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:25.839220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-32455274-0966-4e06-ab75-294dd597c86f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:25.847932 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-32455274-0966-4e06-ab75-294dd597c86f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:25.848603 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-32455274-0966-4e06-ab75-294dd597c86f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:25.849891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-32455274-0966-4e06-ab75-294dd597c86f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:25.850079 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-32455274-0966-4e06-ab75-294dd597c86f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:25.850548 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-32455274-0966-4e06-ab75-294dd597c86f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:25.851194 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-32455274-0966-4e06-ab75-294dd597c86f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:25.905738 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-32455274-0966-4e06-ab75-294dd597c86f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.075725 Sep 08 14:01:25.906756 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 484/1077] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:25 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:25.915304 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-388a94b2-d6e9-40af-97a6-9886ea6ece3e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:25.918041 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-388a94b2-d6e9-40af-97a6-9886ea6ece3e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:25.927613 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-388a94b2-d6e9-40af-97a6-9886ea6ece3e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:25.928108 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-388a94b2-d6e9-40af-97a6-9886ea6ece3e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:25.928546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-388a94b2-d6e9-40af-97a6-9886ea6ece3e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:25.929253 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-388a94b2-d6e9-40af-97a6-9886ea6ece3e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:25.929716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-388a94b2-d6e9-40af-97a6-9886ea6ece3e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:25.931000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-388a94b2-d6e9-40af-97a6-9886ea6ece3e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:25.952246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:25.972994 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-388a94b2-d6e9-40af-97a6-9886ea6ece3e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063218 Sep 08 14:01:25.973657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 594/1078] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:25 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:26.379119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c0df6e43-3d2b-4fc7-8e33-38e793eb6952 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:26.382747 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c0df6e43-3d2b-4fc7-8e33-38e793eb6952 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:26.393744 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c0df6e43-3d2b-4fc7-8e33-38e793eb6952 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:26.394663 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c0df6e43-3d2b-4fc7-8e33-38e793eb6952 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:26.395449 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c0df6e43-3d2b-4fc7-8e33-38e793eb6952 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:26.396723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c0df6e43-3d2b-4fc7-8e33-38e793eb6952 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:26.397557 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c0df6e43-3d2b-4fc7-8e33-38e793eb6952 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:26.398393 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c0df6e43-3d2b-4fc7-8e33-38e793eb6952 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:26.449994 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c0df6e43-3d2b-4fc7-8e33-38e793eb6952 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.069010 Sep 08 14:01:26.449994 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 485/1079] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:26 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:26.926061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-08f10620-3afe-4c71-8669-af757a9aa768 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:26.929628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-08f10620-3afe-4c71-8669-af757a9aa768 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:26.941704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08f10620-3afe-4c71-8669-af757a9aa768 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:26.942653 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08f10620-3afe-4c71-8669-af757a9aa768 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:26.943375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08f10620-3afe-4c71-8669-af757a9aa768 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:26.944578 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08f10620-3afe-4c71-8669-af757a9aa768 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:26.945333 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08f10620-3afe-4c71-8669-af757a9aa768 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:26.945974 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-08f10620-3afe-4c71-8669-af757a9aa768 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:26.999268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-08f10620-3afe-4c71-8669-af757a9aa768 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.079110 Sep 08 14:01:27.001275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 595/1080] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:26 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:27.001449 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-aa5ca730-7b6c-40b3-b25d-b725317026fe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:27.004149 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-aa5ca730-7b6c-40b3-b25d-b725317026fe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:27.011444 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa5ca730-7b6c-40b3-b25d-b725317026fe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:27.012061 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa5ca730-7b6c-40b3-b25d-b725317026fe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:27.012786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa5ca730-7b6c-40b3-b25d-b725317026fe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:27.013079 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa5ca730-7b6c-40b3-b25d-b725317026fe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:27.013461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa5ca730-7b6c-40b3-b25d-b725317026fe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:27.014095 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa5ca730-7b6c-40b3-b25d-b725317026fe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:27.056134 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-aa5ca730-7b6c-40b3-b25d-b725317026fe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059346 Sep 08 14:01:27.057078 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 486/1081] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:26 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:27.094064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2f6eb502-2fd2-467e-aecf-a6b4ce9394dd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:27.094746 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2f6eb502-2fd2-467e-aecf-a6b4ce9394dd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:27.095367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-59a6a03e-e158-487e-8f0c-29dadeff30d0 req-673144af-518d-422c-bf95-6d3488cfb517 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6c3f043c-0662-4b63-bbd5-3a0f68c89b94", "tag": "0476c717-e075-4f94-b8e8-3fa5f88138b1"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:01:27.104190 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f6eb502-2fd2-467e-aecf-a6b4ce9394dd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:27.105752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f6eb502-2fd2-467e-aecf-a6b4ce9394dd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:27.106628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f6eb502-2fd2-467e-aecf-a6b4ce9394dd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:27.107914 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f6eb502-2fd2-467e-aecf-a6b4ce9394dd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:27.109096 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f6eb502-2fd2-467e-aecf-a6b4ce9394dd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:27.114111 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f6eb502-2fd2-467e-aecf-a6b4ce9394dd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:27.114399 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-59a6a03e-e158-487e-8f0c-29dadeff30d0 req-673144af-518d-422c-bf95-6d3488cfb517 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:27.114399 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-59a6a03e-e158-487e-8f0c-29dadeff30d0 req-673144af-518d-422c-bf95-6d3488cfb517 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:27.114719 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-59a6a03e-e158-487e-8f0c-29dadeff30d0 req-673144af-518d-422c-bf95-6d3488cfb517 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:27.129821 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-59a6a03e-e158-487e-8f0c-29dadeff30d0 req-673144af-518d-422c-bf95-6d3488cfb517 service nova] Creating event network-changed:0476c717-e075-4f94-b8e8-3fa5f88138b1 for instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 on hv-cinder-85646 Sep 08 14:01:27.139717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-59a6a03e-e158-487e-8f0c-29dadeff30d0 req-673144af-518d-422c-bf95-6d3488cfb517 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.048371 Sep 08 14:01:27.140332 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 487/1082] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:01:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:27.162255 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2f6eb502-2fd2-467e-aecf-a6b4ce9394dd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:01:27.461855 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-375e81c2-8cc8-41ac-857e-caf2cbd22232 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:27.464959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-375e81c2-8cc8-41ac-857e-caf2cbd22232 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:27.472508 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-375e81c2-8cc8-41ac-857e-caf2cbd22232 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:27.473903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-375e81c2-8cc8-41ac-857e-caf2cbd22232 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:27.476152 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-375e81c2-8cc8-41ac-857e-caf2cbd22232 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:27.476152 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-375e81c2-8cc8-41ac-857e-caf2cbd22232 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:27.480913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-375e81c2-8cc8-41ac-857e-caf2cbd22232 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:27.480913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-375e81c2-8cc8-41ac-857e-caf2cbd22232 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:27.523731 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2f6eb502-2fd2-467e-aecf-a6b4ce9394dd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "DELETE /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 204 len: 0 microversion: 2.1 time: 0.433656 Sep 08 14:01:27.529877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 596/1083] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:01:27 2022] DELETE /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:01:27.530298 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-375e81c2-8cc8-41ac-857e-caf2cbd22232 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.071377 Sep 08 14:01:27.530298 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 488/1084] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:27 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:27.539267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-38b44154-9135-460a-837e-cd9347d95bd0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:27.540417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-38b44154-9135-460a-837e-cd9347d95bd0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:27.550762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38b44154-9135-460a-837e-cd9347d95bd0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:27.551675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38b44154-9135-460a-837e-cd9347d95bd0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:27.552405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38b44154-9135-460a-837e-cd9347d95bd0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:27.553606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38b44154-9135-460a-837e-cd9347d95bd0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:27.554489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38b44154-9135-460a-837e-cd9347d95bd0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:27.555794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38b44154-9135-460a-837e-cd9347d95bd0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:27.792319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-38b44154-9135-460a-837e-cd9347d95bd0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:27.803096 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-38b44154-9135-460a-837e-cd9347d95bd0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1575 microversion: 2.1 time: 0.270977 Sep 08 14:01:27.803693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 597/1085] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:27 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1575 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:01:28.017569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a44ed932-cc7b-4025-95d8-9a334773391a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:28.020847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a44ed932-cc7b-4025-95d8-9a334773391a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:28.038576 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a44ed932-cc7b-4025-95d8-9a334773391a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:28.038971 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a44ed932-cc7b-4025-95d8-9a334773391a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:28.039456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a44ed932-cc7b-4025-95d8-9a334773391a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:28.040176 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a44ed932-cc7b-4025-95d8-9a334773391a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:28.040627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a44ed932-cc7b-4025-95d8-9a334773391a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:28.041115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a44ed932-cc7b-4025-95d8-9a334773391a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:28.068746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:28.076329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-27408cac-5d02-4df8-83d1-a8cd8e88b749 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:28.079397 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-27408cac-5d02-4df8-83d1-a8cd8e88b749 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:28.088028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a44ed932-cc7b-4025-95d8-9a334773391a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.076914 Sep 08 14:01:28.088891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 489/1086] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:28 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:28.090841 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27408cac-5d02-4df8-83d1-a8cd8e88b749 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:28.091677 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27408cac-5d02-4df8-83d1-a8cd8e88b749 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:28.092127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27408cac-5d02-4df8-83d1-a8cd8e88b749 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:28.092874 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27408cac-5d02-4df8-83d1-a8cd8e88b749 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:28.093563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27408cac-5d02-4df8-83d1-a8cd8e88b749 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:28.096725 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-27408cac-5d02-4df8-83d1-a8cd8e88b749 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:28.133931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-27408cac-5d02-4df8-83d1-a8cd8e88b749 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064195 Sep 08 14:01:28.134823 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 598/1087] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:28 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:28.546854 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-942e0331-0698-4a18-abae-dc27511cc151 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:28.550448 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-942e0331-0698-4a18-abae-dc27511cc151 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:28.562535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-942e0331-0698-4a18-abae-dc27511cc151 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:28.563335 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-942e0331-0698-4a18-abae-dc27511cc151 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:28.564078 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-942e0331-0698-4a18-abae-dc27511cc151 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:28.565314 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-942e0331-0698-4a18-abae-dc27511cc151 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:28.565850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-942e0331-0698-4a18-abae-dc27511cc151 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:28.566513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-942e0331-0698-4a18-abae-dc27511cc151 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:28.622922 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-942e0331-0698-4a18-abae-dc27511cc151 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.082690 Sep 08 14:01:28.623533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 490/1088] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:28 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:01:28.837388 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-286e4194-8180-40eb-97d3-5ec70ea2657f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:28.838464 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-286e4194-8180-40eb-97d3-5ec70ea2657f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:28.848456 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-286e4194-8180-40eb-97d3-5ec70ea2657f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:28.849246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-286e4194-8180-40eb-97d3-5ec70ea2657f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:28.849952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-286e4194-8180-40eb-97d3-5ec70ea2657f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:28.851160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-286e4194-8180-40eb-97d3-5ec70ea2657f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:28.852491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-286e4194-8180-40eb-97d3-5ec70ea2657f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:28.853203 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-286e4194-8180-40eb-97d3-5ec70ea2657f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:29.063721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-286e4194-8180-40eb-97d3-5ec70ea2657f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:29.072365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-286e4194-8180-40eb-97d3-5ec70ea2657f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1575 microversion: 2.1 time: 0.241544 Sep 08 14:01:29.073028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 599/1089] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:28 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1575 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:29.108131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-803c06dd-294f-4303-a006-40aa1728b3fb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:29.111564 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-803c06dd-294f-4303-a006-40aa1728b3fb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:29.121549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-803c06dd-294f-4303-a006-40aa1728b3fb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:29.122397 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-803c06dd-294f-4303-a006-40aa1728b3fb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:29.123133 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-803c06dd-294f-4303-a006-40aa1728b3fb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:29.124320 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-803c06dd-294f-4303-a006-40aa1728b3fb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:29.125046 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-803c06dd-294f-4303-a006-40aa1728b3fb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:29.125598 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-803c06dd-294f-4303-a006-40aa1728b3fb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:29.151260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fdda54f0-0cbc-4b96-a54e-6abdfd1a687b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:29.154438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fdda54f0-0cbc-4b96-a54e-6abdfd1a687b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:29.163933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fdda54f0-0cbc-4b96-a54e-6abdfd1a687b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:29.165124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fdda54f0-0cbc-4b96-a54e-6abdfd1a687b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:29.165922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fdda54f0-0cbc-4b96-a54e-6abdfd1a687b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:29.166979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fdda54f0-0cbc-4b96-a54e-6abdfd1a687b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:29.169615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fdda54f0-0cbc-4b96-a54e-6abdfd1a687b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:29.169615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fdda54f0-0cbc-4b96-a54e-6abdfd1a687b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:29.170000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-803c06dd-294f-4303-a006-40aa1728b3fb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067243 Sep 08 14:01:29.174081 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 491/1090] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:29 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:29.214475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fdda54f0-0cbc-4b96-a54e-6abdfd1a687b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.070791 Sep 08 14:01:29.215514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 600/1091] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:29 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:29.640652 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-187ff83c-0957-43f2-9c70-245a4a8fef29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:29.642914 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-187ff83c-0957-43f2-9c70-245a4a8fef29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:29.659086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-187ff83c-0957-43f2-9c70-245a4a8fef29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:29.659601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-187ff83c-0957-43f2-9c70-245a4a8fef29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:29.660258 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-187ff83c-0957-43f2-9c70-245a4a8fef29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:29.661295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-187ff83c-0957-43f2-9c70-245a4a8fef29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:29.661762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-187ff83c-0957-43f2-9c70-245a4a8fef29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:29.662224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-187ff83c-0957-43f2-9c70-245a4a8fef29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:29.699479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-187ff83c-0957-43f2-9c70-245a4a8fef29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.064066 Sep 08 14:01:29.700139 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 492/1092] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:29 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:30.100738 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c0338fba-c1ae-4e6d-9c16-b829beea42dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:30.102357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c0338fba-c1ae-4e6d-9c16-b829beea42dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:30.110594 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0338fba-c1ae-4e6d-9c16-b829beea42dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:30.111198 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0338fba-c1ae-4e6d-9c16-b829beea42dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:30.111712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0338fba-c1ae-4e6d-9c16-b829beea42dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:30.112561 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0338fba-c1ae-4e6d-9c16-b829beea42dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:30.113383 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0338fba-c1ae-4e6d-9c16-b829beea42dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:30.114341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0338fba-c1ae-4e6d-9c16-b829beea42dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:30.186042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-57896d7d-1782-4e97-bc2c-6cf692f733c8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:30.189882 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-57896d7d-1782-4e97-bc2c-6cf692f733c8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:30.198538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-57896d7d-1782-4e97-bc2c-6cf692f733c8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:30.199045 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-57896d7d-1782-4e97-bc2c-6cf692f733c8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:30.199903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-57896d7d-1782-4e97-bc2c-6cf692f733c8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:30.200997 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-57896d7d-1782-4e97-bc2c-6cf692f733c8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:30.201839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-57896d7d-1782-4e97-bc2c-6cf692f733c8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:30.202623 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-57896d7d-1782-4e97-bc2c-6cf692f733c8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:30.254543 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-57896d7d-1782-4e97-bc2c-6cf692f733c8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.076213 Sep 08 14:01:30.256658 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 493/1093] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:30 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:30.264051 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-450dc347-16e6-4707-b9b8-92cd1db3fcc6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:30.268464 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-450dc347-16e6-4707-b9b8-92cd1db3fcc6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:30.277058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-450dc347-16e6-4707-b9b8-92cd1db3fcc6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:30.278107 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-450dc347-16e6-4707-b9b8-92cd1db3fcc6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:30.278538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-450dc347-16e6-4707-b9b8-92cd1db3fcc6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:30.279435 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-450dc347-16e6-4707-b9b8-92cd1db3fcc6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:30.280055 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-450dc347-16e6-4707-b9b8-92cd1db3fcc6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:30.281110 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-450dc347-16e6-4707-b9b8-92cd1db3fcc6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:30.336147 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c0338fba-c1ae-4e6d-9c16-b829beea42dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:30.344933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-450dc347-16e6-4707-b9b8-92cd1db3fcc6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.087086 Sep 08 14:01:30.345649 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 494/1094] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:30 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:30.359816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c0338fba-c1ae-4e6d-9c16-b829beea42dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1456 microversion: 2.1 time: 0.264841 Sep 08 14:01:30.360845 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 601/1095] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:30 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1456 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:30.696933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-7f1180ab-f733-44fc-b829-42269d0a1e50 req-744600c7-1908-481e-bdee-a933607b2852 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6c3f043c-0662-4b63-bbd5-3a0f68c89b94", "name": "network-vif-deleted", "tag": "0476c717-e075-4f94-b8e8-3fa5f88138b1"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:01:30.713931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-7f1180ab-f733-44fc-b829-42269d0a1e50 req-744600c7-1908-481e-bdee-a933607b2852 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:30.714715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-7f1180ab-f733-44fc-b829-42269d0a1e50 req-744600c7-1908-481e-bdee-a933607b2852 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:30.715358 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-7f1180ab-f733-44fc-b829-42269d0a1e50 req-744600c7-1908-481e-bdee-a933607b2852 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:30.719884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cfb14457-5167-4ff3-b565-e9b8acc06351 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:30.725125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cfb14457-5167-4ff3-b565-e9b8acc06351 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:30.740024 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-7f1180ab-f733-44fc-b829-42269d0a1e50 req-744600c7-1908-481e-bdee-a933607b2852 service nova] Creating event network-vif-deleted:0476c717-e075-4f94-b8e8-3fa5f88138b1 for instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 on hv-cinder-85646 Sep 08 14:01:30.740588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cfb14457-5167-4ff3-b565-e9b8acc06351 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:30.741079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cfb14457-5167-4ff3-b565-e9b8acc06351 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:30.741930 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cfb14457-5167-4ff3-b565-e9b8acc06351 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:30.743308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cfb14457-5167-4ff3-b565-e9b8acc06351 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:30.744077 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cfb14457-5167-4ff3-b565-e9b8acc06351 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:30.744688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cfb14457-5167-4ff3-b565-e9b8acc06351 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:30.746315 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-7f1180ab-f733-44fc-b829-42269d0a1e50 req-744600c7-1908-481e-bdee-a933607b2852 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055318 Sep 08 14:01:30.750336 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 495/1096] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:01:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:30.786590 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cfb14457-5167-4ff3-b565-e9b8acc06351 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.075739 Sep 08 14:01:30.787542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 602/1097] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:30 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:31.272410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1d88738d-dfcf-4664-a23f-276d97dca1eb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:31.274851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1d88738d-dfcf-4664-a23f-276d97dca1eb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:31.281983 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d88738d-dfcf-4664-a23f-276d97dca1eb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:31.282519 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d88738d-dfcf-4664-a23f-276d97dca1eb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:31.282960 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d88738d-dfcf-4664-a23f-276d97dca1eb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:31.283691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d88738d-dfcf-4664-a23f-276d97dca1eb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:31.284152 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d88738d-dfcf-4664-a23f-276d97dca1eb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:31.284583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d88738d-dfcf-4664-a23f-276d97dca1eb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:31.320434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1d88738d-dfcf-4664-a23f-276d97dca1eb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053719 Sep 08 14:01:31.321022 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 496/1098] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:31 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:31.360257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d1f82996-2a94-4a8e-b543-41200610caf8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:31.364045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d1f82996-2a94-4a8e-b543-41200610caf8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:31.374475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d1f82996-2a94-4a8e-b543-41200610caf8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:31.375389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d1f82996-2a94-4a8e-b543-41200610caf8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:31.375999 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d1f82996-2a94-4a8e-b543-41200610caf8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:31.377225 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d1f82996-2a94-4a8e-b543-41200610caf8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:31.377986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d1f82996-2a94-4a8e-b543-41200610caf8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:31.378756 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d1f82996-2a94-4a8e-b543-41200610caf8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:31.389062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2a750a06-73a6-4d41-a44c-0353d8b9f16d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:31.389666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2a750a06-73a6-4d41-a44c-0353d8b9f16d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:31.397198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2a750a06-73a6-4d41-a44c-0353d8b9f16d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:31.397750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2a750a06-73a6-4d41-a44c-0353d8b9f16d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:31.399116 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2a750a06-73a6-4d41-a44c-0353d8b9f16d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:31.399951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2a750a06-73a6-4d41-a44c-0353d8b9f16d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:31.400414 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2a750a06-73a6-4d41-a44c-0353d8b9f16d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:31.400838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2a750a06-73a6-4d41-a44c-0353d8b9f16d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:31.426053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d1f82996-2a94-4a8e-b543-41200610caf8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.071046 Sep 08 14:01:31.426955 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 603/1099] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:31 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:31.497673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-2a750a06-73a6-4d41-a44c-0353d8b9f16d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:31.506089 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2a750a06-73a6-4d41-a44c-0353d8b9f16d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.121413 Sep 08 14:01:31.506857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 497/1100] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:31 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:31.801838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f8c75950-5980-4b12-ae8a-e13a3387008d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:31.804349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f8c75950-5980-4b12-ae8a-e13a3387008d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:31.814112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8c75950-5980-4b12-ae8a-e13a3387008d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:31.815001 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8c75950-5980-4b12-ae8a-e13a3387008d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:31.815757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8c75950-5980-4b12-ae8a-e13a3387008d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:31.816991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8c75950-5980-4b12-ae8a-e13a3387008d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:31.817816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8c75950-5980-4b12-ae8a-e13a3387008d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:31.818541 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8c75950-5980-4b12-ae8a-e13a3387008d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:31.863660 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f8c75950-5980-4b12-ae8a-e13a3387008d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.065535 Sep 08 14:01:31.864685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 604/1101] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:31 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:32.335371 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e0b43bdd-c726-4419-8a13-c065f9bf08af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:32.338244 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e0b43bdd-c726-4419-8a13-c065f9bf08af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:32.345543 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e0b43bdd-c726-4419-8a13-c065f9bf08af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:32.346041 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e0b43bdd-c726-4419-8a13-c065f9bf08af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:32.346519 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e0b43bdd-c726-4419-8a13-c065f9bf08af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:32.347250 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e0b43bdd-c726-4419-8a13-c065f9bf08af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:32.347703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e0b43bdd-c726-4419-8a13-c065f9bf08af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:32.348124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e0b43bdd-c726-4419-8a13-c065f9bf08af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:32.405381 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e0b43bdd-c726-4419-8a13-c065f9bf08af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.074417 Sep 08 14:01:32.405985 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 498/1102] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:32 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:32.444715 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-85910cae-ce48-485e-a47b-2a6258a4e427 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:32.447699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-85910cae-ce48-485e-a47b-2a6258a4e427 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:32.454358 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85910cae-ce48-485e-a47b-2a6258a4e427 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:32.455529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85910cae-ce48-485e-a47b-2a6258a4e427 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:32.455960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85910cae-ce48-485e-a47b-2a6258a4e427 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:32.457466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85910cae-ce48-485e-a47b-2a6258a4e427 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:32.458533 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85910cae-ce48-485e-a47b-2a6258a4e427 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:32.459073 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85910cae-ce48-485e-a47b-2a6258a4e427 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:32.498383 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-85910cae-ce48-485e-a47b-2a6258a4e427 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059399 Sep 08 14:01:32.498969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 605/1103] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:32 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:32.751893 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-84f14117-2598-4f7a-95e3-515786a59231 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:32.752642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-84f14117-2598-4f7a-95e3-515786a59231 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:32.762635 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84f14117-2598-4f7a-95e3-515786a59231 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:32.763132 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84f14117-2598-4f7a-95e3-515786a59231 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:32.763597 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84f14117-2598-4f7a-95e3-515786a59231 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:32.764312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84f14117-2598-4f7a-95e3-515786a59231 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:32.764762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84f14117-2598-4f7a-95e3-515786a59231 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:32.765216 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84f14117-2598-4f7a-95e3-515786a59231 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:32.864942 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-84f14117-2598-4f7a-95e3-515786a59231 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:32.874793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-84f14117-2598-4f7a-95e3-515786a59231 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.345028 Sep 08 14:01:32.875664 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 499/1104] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:32 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:32.886914 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-aa7931ab-15e7-41e0-97ed-89f0f95bde89 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:32.890050 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-aa7931ab-15e7-41e0-97ed-89f0f95bde89 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:32.898444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa7931ab-15e7-41e0-97ed-89f0f95bde89 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:32.898969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa7931ab-15e7-41e0-97ed-89f0f95bde89 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:32.899410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa7931ab-15e7-41e0-97ed-89f0f95bde89 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:32.900117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa7931ab-15e7-41e0-97ed-89f0f95bde89 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:32.900639 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa7931ab-15e7-41e0-97ed-89f0f95bde89 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:32.901063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa7931ab-15e7-41e0-97ed-89f0f95bde89 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:32.942621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-aa7931ab-15e7-41e0-97ed-89f0f95bde89 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.062071 Sep 08 14:01:32.943282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 606/1105] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:32 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:33.418662 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7813d51a-d2ed-41c9-885b-60a971dc6fbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:33.421147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7813d51a-d2ed-41c9-885b-60a971dc6fbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:33.429754 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7813d51a-d2ed-41c9-885b-60a971dc6fbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:33.430573 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7813d51a-d2ed-41c9-885b-60a971dc6fbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:33.431101 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7813d51a-d2ed-41c9-885b-60a971dc6fbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:33.431833 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7813d51a-d2ed-41c9-885b-60a971dc6fbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:33.432297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7813d51a-d2ed-41c9-885b-60a971dc6fbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:33.432728 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7813d51a-d2ed-41c9-885b-60a971dc6fbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:33.478194 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7813d51a-d2ed-41c9-885b-60a971dc6fbd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063390 Sep 08 14:01:33.478801 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 500/1106] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:33 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:33.515028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-07ae9ec1-93f4-4d54-927f-534f3d4df8e4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:33.517179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-07ae9ec1-93f4-4d54-927f-534f3d4df8e4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:33.525010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-07ae9ec1-93f4-4d54-927f-534f3d4df8e4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:33.525505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-07ae9ec1-93f4-4d54-927f-534f3d4df8e4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:33.525943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-07ae9ec1-93f4-4d54-927f-534f3d4df8e4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:33.526774 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-07ae9ec1-93f4-4d54-927f-534f3d4df8e4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:33.527220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-07ae9ec1-93f4-4d54-927f-534f3d4df8e4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:33.527639 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-07ae9ec1-93f4-4d54-927f-534f3d4df8e4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:33.578529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-07ae9ec1-93f4-4d54-927f-534f3d4df8e4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067482 Sep 08 14:01:33.579137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 607/1107] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:33 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:33.911698 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c6c9bab7-906b-4030-b482-ab6d017d1b4d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:33.912366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c6c9bab7-906b-4030-b482-ab6d017d1b4d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:33.919855 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6c9bab7-906b-4030-b482-ab6d017d1b4d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:33.920338 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6c9bab7-906b-4030-b482-ab6d017d1b4d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:33.920861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6c9bab7-906b-4030-b482-ab6d017d1b4d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:33.921685 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6c9bab7-906b-4030-b482-ab6d017d1b4d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:33.922251 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6c9bab7-906b-4030-b482-ab6d017d1b4d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:33.922825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6c9bab7-906b-4030-b482-ab6d017d1b4d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:33.958548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2f7f25e7-9aba-4006-8f9b-57d8277a5bf9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:33.962222 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2f7f25e7-9aba-4006-8f9b-57d8277a5bf9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:33.972064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f7f25e7-9aba-4006-8f9b-57d8277a5bf9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:33.972895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f7f25e7-9aba-4006-8f9b-57d8277a5bf9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:33.973640 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f7f25e7-9aba-4006-8f9b-57d8277a5bf9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:33.975061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f7f25e7-9aba-4006-8f9b-57d8277a5bf9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:33.975657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f7f25e7-9aba-4006-8f9b-57d8277a5bf9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:33.976937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f7f25e7-9aba-4006-8f9b-57d8277a5bf9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:34.008598 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c6c9bab7-906b-4030-b482-ab6d017d1b4d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:34.019114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c6c9bab7-906b-4030-b482-ab6d017d1b4d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.111745 Sep 08 14:01:34.020175 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 501/1108] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:33 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:34.025601 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2f7f25e7-9aba-4006-8f9b-57d8277a5bf9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.072935 Sep 08 14:01:34.026848 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 608/1109] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:33 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:01:34.495486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-99d8c560-03df-402c-8bdd-cc40aebab9e5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:34.498240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-99d8c560-03df-402c-8bdd-cc40aebab9e5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:34.505640 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-99d8c560-03df-402c-8bdd-cc40aebab9e5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:34.506307 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-99d8c560-03df-402c-8bdd-cc40aebab9e5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:34.506955 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-99d8c560-03df-402c-8bdd-cc40aebab9e5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:34.507905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-99d8c560-03df-402c-8bdd-cc40aebab9e5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:34.508981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-99d8c560-03df-402c-8bdd-cc40aebab9e5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:34.509403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-99d8c560-03df-402c-8bdd-cc40aebab9e5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:34.548852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-99d8c560-03df-402c-8bdd-cc40aebab9e5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057507 Sep 08 14:01:34.549564 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 502/1110] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:34 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:34.594765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b602862a-77da-4b9c-be85-f3157a0e3f73 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:34.598316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b602862a-77da-4b9c-be85-f3157a0e3f73 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:34.608724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b602862a-77da-4b9c-be85-f3157a0e3f73 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:34.609287 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b602862a-77da-4b9c-be85-f3157a0e3f73 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:34.609727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b602862a-77da-4b9c-be85-f3157a0e3f73 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:34.610739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b602862a-77da-4b9c-be85-f3157a0e3f73 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:34.612603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b602862a-77da-4b9c-be85-f3157a0e3f73 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:34.612603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b602862a-77da-4b9c-be85-f3157a0e3f73 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:34.638174 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:34.658579 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b602862a-77da-4b9c-be85-f3157a0e3f73 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069989 Sep 08 14:01:34.659185 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 609/1111] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:34 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:35.040352 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c2197244-66ba-40b4-aa44-d86022f49331 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:35.042968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c2197244-66ba-40b4-aa44-d86022f49331 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:35.049696 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2197244-66ba-40b4-aa44-d86022f49331 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:35.050221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2197244-66ba-40b4-aa44-d86022f49331 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:35.050662 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2197244-66ba-40b4-aa44-d86022f49331 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:35.051370 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2197244-66ba-40b4-aa44-d86022f49331 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:35.051853 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2197244-66ba-40b4-aa44-d86022f49331 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:35.052294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2197244-66ba-40b4-aa44-d86022f49331 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:35.056313 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-14e61110-c18e-4b55-b51e-90ae5e3ef3f1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:35.056977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-14e61110-c18e-4b55-b51e-90ae5e3ef3f1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:35.064803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-14e61110-c18e-4b55-b51e-90ae5e3ef3f1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:35.065303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-14e61110-c18e-4b55-b51e-90ae5e3ef3f1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:35.065745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-14e61110-c18e-4b55-b51e-90ae5e3ef3f1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:35.066589 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-14e61110-c18e-4b55-b51e-90ae5e3ef3f1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:35.067008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-14e61110-c18e-4b55-b51e-90ae5e3ef3f1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:35.067443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-14e61110-c18e-4b55-b51e-90ae5e3ef3f1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:35.096615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c2197244-66ba-40b4-aa44-d86022f49331 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.060792 Sep 08 14:01:35.097238 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 503/1112] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:35 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:01:35.153628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-14e61110-c18e-4b55-b51e-90ae5e3ef3f1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:35.164936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-14e61110-c18e-4b55-b51e-90ae5e3ef3f1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.113703 Sep 08 14:01:35.167402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 610/1113] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:35 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:01:35.564768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1d56c09c-5509-4aed-a13e-b3bacce28c35 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:35.568303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1d56c09c-5509-4aed-a13e-b3bacce28c35 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:35.579271 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d56c09c-5509-4aed-a13e-b3bacce28c35 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:35.580082 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d56c09c-5509-4aed-a13e-b3bacce28c35 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:35.580783 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d56c09c-5509-4aed-a13e-b3bacce28c35 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:35.581922 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d56c09c-5509-4aed-a13e-b3bacce28c35 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:35.582735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d56c09c-5509-4aed-a13e-b3bacce28c35 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:35.583980 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d56c09c-5509-4aed-a13e-b3bacce28c35 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:35.629112 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1d56c09c-5509-4aed-a13e-b3bacce28c35 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.071236 Sep 08 14:01:35.629698 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 504/1114] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:35 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:35.674108 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4e75f761-dcae-4a0b-ae89-05531bd4bedb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:35.676292 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4e75f761-dcae-4a0b-ae89-05531bd4bedb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:35.683644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4e75f761-dcae-4a0b-ae89-05531bd4bedb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:35.684140 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4e75f761-dcae-4a0b-ae89-05531bd4bedb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:35.684577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4e75f761-dcae-4a0b-ae89-05531bd4bedb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:35.685284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4e75f761-dcae-4a0b-ae89-05531bd4bedb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:35.685733 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4e75f761-dcae-4a0b-ae89-05531bd4bedb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:35.686165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4e75f761-dcae-4a0b-ae89-05531bd4bedb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:35.723158 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4e75f761-dcae-4a0b-ae89-05531bd4bedb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052969 Sep 08 14:01:35.723741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 611/1115] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:35 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:36.111332 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-360dbf56-880f-4870-8a38-74b3d75da5f8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:36.113429 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-360dbf56-880f-4870-8a38-74b3d75da5f8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:36.119903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-360dbf56-880f-4870-8a38-74b3d75da5f8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:36.120404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-360dbf56-880f-4870-8a38-74b3d75da5f8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:36.120833 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-360dbf56-880f-4870-8a38-74b3d75da5f8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:36.121538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-360dbf56-880f-4870-8a38-74b3d75da5f8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:36.121988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-360dbf56-880f-4870-8a38-74b3d75da5f8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:36.122448 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-360dbf56-880f-4870-8a38-74b3d75da5f8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:36.156039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-360dbf56-880f-4870-8a38-74b3d75da5f8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049078 Sep 08 14:01:36.156631 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 505/1116] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:36 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:36.199526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ca2a3396-486d-4f25-9d23-f6399685dc11 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:36.200132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ca2a3396-486d-4f25-9d23-f6399685dc11 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:36.206676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca2a3396-486d-4f25-9d23-f6399685dc11 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:36.207221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca2a3396-486d-4f25-9d23-f6399685dc11 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:36.207702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca2a3396-486d-4f25-9d23-f6399685dc11 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:36.208474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca2a3396-486d-4f25-9d23-f6399685dc11 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:36.209080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca2a3396-486d-4f25-9d23-f6399685dc11 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:36.209557 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca2a3396-486d-4f25-9d23-f6399685dc11 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:36.281232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ca2a3396-486d-4f25-9d23-f6399685dc11 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:36.289820 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ca2a3396-486d-4f25-9d23-f6399685dc11 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.094114 Sep 08 14:01:36.290445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 612/1117] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:36 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:36.642864 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a0db78af-3057-4050-b03c-dcec483a4bc5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:36.645716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a0db78af-3057-4050-b03c-dcec483a4bc5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:36.654767 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0db78af-3057-4050-b03c-dcec483a4bc5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:36.655248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0db78af-3057-4050-b03c-dcec483a4bc5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:36.655678 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0db78af-3057-4050-b03c-dcec483a4bc5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:36.656380 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0db78af-3057-4050-b03c-dcec483a4bc5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:36.656825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0db78af-3057-4050-b03c-dcec483a4bc5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:36.657238 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0db78af-3057-4050-b03c-dcec483a4bc5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:36.685693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:36.690687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a0db78af-3057-4050-b03c-dcec483a4bc5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052521 Sep 08 14:01:36.691313 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 506/1118] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:36 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:36.736003 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-dedaa505-39bd-4940-bfbb-5843e59c9dda tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:36.738247 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-dedaa505-39bd-4940-bfbb-5843e59c9dda tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:36.747492 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dedaa505-39bd-4940-bfbb-5843e59c9dda tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:36.747977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dedaa505-39bd-4940-bfbb-5843e59c9dda tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:36.748412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dedaa505-39bd-4940-bfbb-5843e59c9dda tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:36.749137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dedaa505-39bd-4940-bfbb-5843e59c9dda tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:36.749582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dedaa505-39bd-4940-bfbb-5843e59c9dda tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:36.750029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dedaa505-39bd-4940-bfbb-5843e59c9dda tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:36.783058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-dedaa505-39bd-4940-bfbb-5843e59c9dda tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051146 Sep 08 14:01:36.783640 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 613/1119] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:36 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:37.169647 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4c30285e-fd1f-4540-b613-a32fd5aa4f47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:37.171976 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4c30285e-fd1f-4540-b613-a32fd5aa4f47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:37.178719 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4c30285e-fd1f-4540-b613-a32fd5aa4f47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:37.179209 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4c30285e-fd1f-4540-b613-a32fd5aa4f47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:37.179751 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4c30285e-fd1f-4540-b613-a32fd5aa4f47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:37.180354 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4c30285e-fd1f-4540-b613-a32fd5aa4f47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:37.180804 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4c30285e-fd1f-4540-b613-a32fd5aa4f47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:37.181230 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4c30285e-fd1f-4540-b613-a32fd5aa4f47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:37.215023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4c30285e-fd1f-4540-b613-a32fd5aa4f47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049835 Sep 08 14:01:37.215606 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 507/1120] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:37 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:01:37.313799 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-62125fce-4e27-4ef3-b26b-accfc199df17 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:37.314506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-62125fce-4e27-4ef3-b26b-accfc199df17 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:37.321503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-62125fce-4e27-4ef3-b26b-accfc199df17 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:37.322001 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-62125fce-4e27-4ef3-b26b-accfc199df17 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:37.322473 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-62125fce-4e27-4ef3-b26b-accfc199df17 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:37.324055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-62125fce-4e27-4ef3-b26b-accfc199df17 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:37.324055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-62125fce-4e27-4ef3-b26b-accfc199df17 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:37.324055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-62125fce-4e27-4ef3-b26b-accfc199df17 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:37.398398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-62125fce-4e27-4ef3-b26b-accfc199df17 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:37.406481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-62125fce-4e27-4ef3-b26b-accfc199df17 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.097062 Sep 08 14:01:37.407074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 614/1121] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:37 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:37.703935 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c41d9f65-e166-4c87-91f7-f774b0c021d6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:37.706161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c41d9f65-e166-4c87-91f7-f774b0c021d6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:37.714316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c41d9f65-e166-4c87-91f7-f774b0c021d6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:37.714817 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c41d9f65-e166-4c87-91f7-f774b0c021d6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:37.715262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c41d9f65-e166-4c87-91f7-f774b0c021d6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:37.715981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c41d9f65-e166-4c87-91f7-f774b0c021d6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:37.716436 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c41d9f65-e166-4c87-91f7-f774b0c021d6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:37.716874 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c41d9f65-e166-4c87-91f7-f774b0c021d6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:37.749672 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c41d9f65-e166-4c87-91f7-f774b0c021d6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049731 Sep 08 14:01:37.750292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 508/1122] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:37 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:37.796662 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-97fcd946-ba22-4c7c-b3b2-c04d50c9bebc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:37.799119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-97fcd946-ba22-4c7c-b3b2-c04d50c9bebc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:37.806090 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-97fcd946-ba22-4c7c-b3b2-c04d50c9bebc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:37.806972 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-97fcd946-ba22-4c7c-b3b2-c04d50c9bebc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:37.807757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-97fcd946-ba22-4c7c-b3b2-c04d50c9bebc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:37.809082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-97fcd946-ba22-4c7c-b3b2-c04d50c9bebc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:37.809914 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-97fcd946-ba22-4c7c-b3b2-c04d50c9bebc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:37.810742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-97fcd946-ba22-4c7c-b3b2-c04d50c9bebc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:37.844570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-97fcd946-ba22-4c7c-b3b2-c04d50c9bebc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052589 Sep 08 14:01:37.845136 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 615/1123] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:37 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:38.227850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8c06e760-16b5-4ae6-b899-072a617e48d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:38.229994 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8c06e760-16b5-4ae6-b899-072a617e48d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:38.236190 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8c06e760-16b5-4ae6-b899-072a617e48d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:38.236673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8c06e760-16b5-4ae6-b899-072a617e48d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:38.237103 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8c06e760-16b5-4ae6-b899-072a617e48d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:38.237816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8c06e760-16b5-4ae6-b899-072a617e48d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:38.238301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8c06e760-16b5-4ae6-b899-072a617e48d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:38.238911 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8c06e760-16b5-4ae6-b899-072a617e48d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:38.271595 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8c06e760-16b5-4ae6-b899-072a617e48d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047776 Sep 08 14:01:38.272513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 509/1124] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:38 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:38.432233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a7691a79-f70d-4c7a-9dac-53d0f7fab9a9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:38.432863 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a7691a79-f70d-4c7a-9dac-53d0f7fab9a9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:38.438950 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7691a79-f70d-4c7a-9dac-53d0f7fab9a9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:38.439349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7691a79-f70d-4c7a-9dac-53d0f7fab9a9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:38.439764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7691a79-f70d-4c7a-9dac-53d0f7fab9a9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:38.440449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7691a79-f70d-4c7a-9dac-53d0f7fab9a9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:38.440896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7691a79-f70d-4c7a-9dac-53d0f7fab9a9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:38.441314 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7691a79-f70d-4c7a-9dac-53d0f7fab9a9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:38.502731 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a7691a79-f70d-4c7a-9dac-53d0f7fab9a9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:38.511059 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a7691a79-f70d-4c7a-9dac-53d0f7fab9a9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.083404 Sep 08 14:01:38.511627 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 616/1125] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:38 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:38.763224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-49330808-77de-44cd-b32f-9dd6d8d8fe78 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:38.765249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-49330808-77de-44cd-b32f-9dd6d8d8fe78 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:38.771823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-49330808-77de-44cd-b32f-9dd6d8d8fe78 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:38.772301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-49330808-77de-44cd-b32f-9dd6d8d8fe78 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:38.773273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-49330808-77de-44cd-b32f-9dd6d8d8fe78 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:38.773971 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-49330808-77de-44cd-b32f-9dd6d8d8fe78 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:38.774471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-49330808-77de-44cd-b32f-9dd6d8d8fe78 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:38.774891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-49330808-77de-44cd-b32f-9dd6d8d8fe78 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:38.806010 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-49330808-77de-44cd-b32f-9dd6d8d8fe78 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046760 Sep 08 14:01:38.806588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 510/1126] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:38 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:38.857553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fd4c83e5-3c64-42e6-9850-4288bdb74af8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:38.859791 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fd4c83e5-3c64-42e6-9850-4288bdb74af8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:38.866316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd4c83e5-3c64-42e6-9850-4288bdb74af8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:38.866807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd4c83e5-3c64-42e6-9850-4288bdb74af8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:38.867240 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd4c83e5-3c64-42e6-9850-4288bdb74af8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:38.867953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd4c83e5-3c64-42e6-9850-4288bdb74af8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:38.868401 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd4c83e5-3c64-42e6-9850-4288bdb74af8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:38.868826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd4c83e5-3c64-42e6-9850-4288bdb74af8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:38.899234 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fd4c83e5-3c64-42e6-9850-4288bdb74af8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045585 Sep 08 14:01:38.899869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 617/1127] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:38 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:39.284440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-def49150-6b43-49ed-aad3-0510f368b73c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:39.286613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-def49150-6b43-49ed-aad3-0510f368b73c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:39.292618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-def49150-6b43-49ed-aad3-0510f368b73c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:39.293074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-def49150-6b43-49ed-aad3-0510f368b73c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:39.293501 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-def49150-6b43-49ed-aad3-0510f368b73c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:39.294201 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-def49150-6b43-49ed-aad3-0510f368b73c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:39.294647 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-def49150-6b43-49ed-aad3-0510f368b73c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:39.295059 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-def49150-6b43-49ed-aad3-0510f368b73c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:39.324821 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-def49150-6b43-49ed-aad3-0510f368b73c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044379 Sep 08 14:01:39.325390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 511/1128] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:39 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:39.534422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6c84cc28-ddcb-41bf-9085-4952a00f7aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:39.535125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6c84cc28-ddcb-41bf-9085-4952a00f7aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:39.541392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c84cc28-ddcb-41bf-9085-4952a00f7aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:39.541913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c84cc28-ddcb-41bf-9085-4952a00f7aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:39.542405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c84cc28-ddcb-41bf-9085-4952a00f7aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:39.543107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c84cc28-ddcb-41bf-9085-4952a00f7aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:39.543658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c84cc28-ddcb-41bf-9085-4952a00f7aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:39.544179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c84cc28-ddcb-41bf-9085-4952a00f7aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:39.614744 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6c84cc28-ddcb-41bf-9085-4952a00f7aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:39.623267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6c84cc28-ddcb-41bf-9085-4952a00f7aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.092830 Sep 08 14:01:39.623830 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 618/1129] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:39 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:39.817751 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3e8745bd-2b25-44f3-8da0-90d31f8ec0f4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:39.820465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3e8745bd-2b25-44f3-8da0-90d31f8ec0f4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:39.827137 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e8745bd-2b25-44f3-8da0-90d31f8ec0f4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:39.827626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e8745bd-2b25-44f3-8da0-90d31f8ec0f4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:39.828059 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e8745bd-2b25-44f3-8da0-90d31f8ec0f4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:39.828752 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e8745bd-2b25-44f3-8da0-90d31f8ec0f4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:39.829211 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e8745bd-2b25-44f3-8da0-90d31f8ec0f4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:39.829984 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e8745bd-2b25-44f3-8da0-90d31f8ec0f4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:39.866902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3e8745bd-2b25-44f3-8da0-90d31f8ec0f4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048012 Sep 08 14:01:39.866902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 512/1130] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:39 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:39.912606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a910ad49-aa5b-410a-877c-95163ee46230 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:39.915352 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a910ad49-aa5b-410a-877c-95163ee46230 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:39.921847 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a910ad49-aa5b-410a-877c-95163ee46230 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:39.927508 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a910ad49-aa5b-410a-877c-95163ee46230 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:39.927508 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a910ad49-aa5b-410a-877c-95163ee46230 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:39.927508 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a910ad49-aa5b-410a-877c-95163ee46230 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:39.927508 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a910ad49-aa5b-410a-877c-95163ee46230 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:39.928356 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a910ad49-aa5b-410a-877c-95163ee46230 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:39.958832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a910ad49-aa5b-410a-877c-95163ee46230 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050813 Sep 08 14:01:39.961932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 619/1131] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:39 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:40.337214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c6ac2b3a-a053-4455-ba22-5e4213c9f77d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:40.339388 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c6ac2b3a-a053-4455-ba22-5e4213c9f77d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:40.345605 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6ac2b3a-a053-4455-ba22-5e4213c9f77d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:40.346065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6ac2b3a-a053-4455-ba22-5e4213c9f77d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:40.346535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6ac2b3a-a053-4455-ba22-5e4213c9f77d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:40.347219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6ac2b3a-a053-4455-ba22-5e4213c9f77d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:40.347665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6ac2b3a-a053-4455-ba22-5e4213c9f77d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:40.348078 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6ac2b3a-a053-4455-ba22-5e4213c9f77d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:40.386640 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c6ac2b3a-a053-4455-ba22-5e4213c9f77d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053305 Sep 08 14:01:40.387212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 513/1132] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:40 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:40.647148 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-93b18b8b-9c50-410b-b5d4-3608e9cb7b56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:40.647791 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-93b18b8b-9c50-410b-b5d4-3608e9cb7b56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:40.653943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93b18b8b-9c50-410b-b5d4-3608e9cb7b56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:40.654466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93b18b8b-9c50-410b-b5d4-3608e9cb7b56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:40.654904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93b18b8b-9c50-410b-b5d4-3608e9cb7b56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:40.655616 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93b18b8b-9c50-410b-b5d4-3608e9cb7b56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:40.656053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93b18b8b-9c50-410b-b5d4-3608e9cb7b56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:40.656471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93b18b8b-9c50-410b-b5d4-3608e9cb7b56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:40.719331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-93b18b8b-9c50-410b-b5d4-3608e9cb7b56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:40.727849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-93b18b8b-9c50-410b-b5d4-3608e9cb7b56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.084988 Sep 08 14:01:40.728433 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 620/1133] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:40 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:40.878062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ef53867f-35e2-4a02-98d0-65eaa6c29027 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:40.880272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ef53867f-35e2-4a02-98d0-65eaa6c29027 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:40.887485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef53867f-35e2-4a02-98d0-65eaa6c29027 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:40.887981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef53867f-35e2-4a02-98d0-65eaa6c29027 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:40.888418 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef53867f-35e2-4a02-98d0-65eaa6c29027 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:40.889129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef53867f-35e2-4a02-98d0-65eaa6c29027 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:40.889578 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef53867f-35e2-4a02-98d0-65eaa6c29027 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:40.890196 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ef53867f-35e2-4a02-98d0-65eaa6c29027 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:40.921120 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ef53867f-35e2-4a02-98d0-65eaa6c29027 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047083 Sep 08 14:01:40.921677 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 514/1134] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:40 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:40.971106 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-dca0ef8e-498e-425f-88d5-c5700aed0867 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:40.973180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-dca0ef8e-498e-425f-88d5-c5700aed0867 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:40.980243 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dca0ef8e-498e-425f-88d5-c5700aed0867 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:40.980735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dca0ef8e-498e-425f-88d5-c5700aed0867 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:40.981180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dca0ef8e-498e-425f-88d5-c5700aed0867 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:40.981895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dca0ef8e-498e-425f-88d5-c5700aed0867 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:40.982366 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dca0ef8e-498e-425f-88d5-c5700aed0867 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:40.982795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dca0ef8e-498e-425f-88d5-c5700aed0867 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:41.012852 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-dca0ef8e-498e-425f-88d5-c5700aed0867 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046021 Sep 08 14:01:41.013403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 621/1135] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:40 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:41.399378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-dff46031-fd51-481f-bb86-e2da83e69d2d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:41.401658 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-dff46031-fd51-481f-bb86-e2da83e69d2d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:41.407838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dff46031-fd51-481f-bb86-e2da83e69d2d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:41.408318 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dff46031-fd51-481f-bb86-e2da83e69d2d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:41.408753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dff46031-fd51-481f-bb86-e2da83e69d2d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:41.409521 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dff46031-fd51-481f-bb86-e2da83e69d2d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:41.410003 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dff46031-fd51-481f-bb86-e2da83e69d2d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:41.410453 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dff46031-fd51-481f-bb86-e2da83e69d2d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:41.440229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-dff46031-fd51-481f-bb86-e2da83e69d2d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044906 Sep 08 14:01:41.440783 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 515/1136] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:41 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:41.750502 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-45d7463a-1ee3-40b5-acd1-b87cd8144646 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:41.751027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-45d7463a-1ee3-40b5-acd1-b87cd8144646 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:41.757216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-45d7463a-1ee3-40b5-acd1-b87cd8144646 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:41.757970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-45d7463a-1ee3-40b5-acd1-b87cd8144646 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:41.758435 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-45d7463a-1ee3-40b5-acd1-b87cd8144646 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:41.759124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-45d7463a-1ee3-40b5-acd1-b87cd8144646 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:41.759577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-45d7463a-1ee3-40b5-acd1-b87cd8144646 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:41.760007 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-45d7463a-1ee3-40b5-acd1-b87cd8144646 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:41.821484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-45d7463a-1ee3-40b5-acd1-b87cd8144646 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:41.829284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-45d7463a-1ee3-40b5-acd1-b87cd8144646 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.082829 Sep 08 14:01:41.829841 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 622/1137] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:41 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:41.933507 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fdc30fbc-2f6d-4468-8cd8-cee242158216 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:41.935804 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fdc30fbc-2f6d-4468-8cd8-cee242158216 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:41.938278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:41.943289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fdc30fbc-2f6d-4468-8cd8-cee242158216 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:41.943784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fdc30fbc-2f6d-4468-8cd8-cee242158216 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:41.944219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fdc30fbc-2f6d-4468-8cd8-cee242158216 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:41.944913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fdc30fbc-2f6d-4468-8cd8-cee242158216 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:41.945524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fdc30fbc-2f6d-4468-8cd8-cee242158216 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:41.945905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fdc30fbc-2f6d-4468-8cd8-cee242158216 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:41.980117 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fdc30fbc-2f6d-4468-8cd8-cee242158216 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050341 Sep 08 14:01:41.980708 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 516/1138] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:41 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:42.025817 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-02a79210-6e84-4afb-8703-89cb9df31903 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:42.027951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-02a79210-6e84-4afb-8703-89cb9df31903 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:42.034627 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02a79210-6e84-4afb-8703-89cb9df31903 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:42.035171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02a79210-6e84-4afb-8703-89cb9df31903 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:42.035549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02a79210-6e84-4afb-8703-89cb9df31903 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:42.036257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02a79210-6e84-4afb-8703-89cb9df31903 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:42.036710 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02a79210-6e84-4afb-8703-89cb9df31903 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:42.037129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02a79210-6e84-4afb-8703-89cb9df31903 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:42.067736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-02a79210-6e84-4afb-8703-89cb9df31903 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046659 Sep 08 14:01:42.068307 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 623/1139] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:42 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:42.453153 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a7367672-516a-4d09-9b75-5f0760c00657 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:42.455362 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a7367672-516a-4d09-9b75-5f0760c00657 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:42.462166 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7367672-516a-4d09-9b75-5f0760c00657 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:42.462683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7367672-516a-4d09-9b75-5f0760c00657 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:42.463122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7367672-516a-4d09-9b75-5f0760c00657 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:42.463892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7367672-516a-4d09-9b75-5f0760c00657 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:42.464271 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7367672-516a-4d09-9b75-5f0760c00657 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:42.464698 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7367672-516a-4d09-9b75-5f0760c00657 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:42.494038 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a7367672-516a-4d09-9b75-5f0760c00657 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044912 Sep 08 14:01:42.494675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 517/1140] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:42 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:42.852863 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b5a7cde2-cf87-4c54-a072-6f10a9879e7c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:42.853461 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b5a7cde2-cf87-4c54-a072-6f10a9879e7c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:42.859393 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5a7cde2-cf87-4c54-a072-6f10a9879e7c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:42.859866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5a7cde2-cf87-4c54-a072-6f10a9879e7c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:42.860300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5a7cde2-cf87-4c54-a072-6f10a9879e7c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:42.860983 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5a7cde2-cf87-4c54-a072-6f10a9879e7c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:42.861433 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5a7cde2-cf87-4c54-a072-6f10a9879e7c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:42.861853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5a7cde2-cf87-4c54-a072-6f10a9879e7c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:42.924175 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b5a7cde2-cf87-4c54-a072-6f10a9879e7c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:42.932435 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b5a7cde2-cf87-4c54-a072-6f10a9879e7c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.083459 Sep 08 14:01:42.932997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 624/1141] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:42 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:42.993395 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-aa248609-cbfd-4e0c-a491-44a6a4db40ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:42.995613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-aa248609-cbfd-4e0c-a491-44a6a4db40ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:43.002064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa248609-cbfd-4e0c-a491-44a6a4db40ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:43.002658 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa248609-cbfd-4e0c-a491-44a6a4db40ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:43.003092 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa248609-cbfd-4e0c-a491-44a6a4db40ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:43.003791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa248609-cbfd-4e0c-a491-44a6a4db40ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:43.004275 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa248609-cbfd-4e0c-a491-44a6a4db40ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:43.004684 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa248609-cbfd-4e0c-a491-44a6a4db40ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:43.033965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-aa248609-cbfd-4e0c-a491-44a6a4db40ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045400 Sep 08 14:01:43.034588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 518/1142] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:42 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:43.080060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2d6a3c03-707d-463a-8252-7bad8578bee2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:43.082204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2d6a3c03-707d-463a-8252-7bad8578bee2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:43.088617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d6a3c03-707d-463a-8252-7bad8578bee2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:43.089098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d6a3c03-707d-463a-8252-7bad8578bee2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:43.089529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d6a3c03-707d-463a-8252-7bad8578bee2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:43.090235 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d6a3c03-707d-463a-8252-7bad8578bee2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:43.090689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d6a3c03-707d-463a-8252-7bad8578bee2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:43.091107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d6a3c03-707d-463a-8252-7bad8578bee2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:43.120545 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2d6a3c03-707d-463a-8252-7bad8578bee2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044428 Sep 08 14:01:43.121091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 625/1143] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:43 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:43.507700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b9052933-a19c-4839-81fe-3ff5e3a604b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:43.509878 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b9052933-a19c-4839-81fe-3ff5e3a604b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:43.516196 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b9052933-a19c-4839-81fe-3ff5e3a604b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:43.516679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b9052933-a19c-4839-81fe-3ff5e3a604b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:43.517107 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b9052933-a19c-4839-81fe-3ff5e3a604b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:43.517796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b9052933-a19c-4839-81fe-3ff5e3a604b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:43.518275 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b9052933-a19c-4839-81fe-3ff5e3a604b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:43.518704 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b9052933-a19c-4839-81fe-3ff5e3a604b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:43.548982 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b9052933-a19c-4839-81fe-3ff5e3a604b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045699 Sep 08 14:01:43.549552 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 519/1144] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:43 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:43.954342 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-05b3c82d-d8d5-49f0-b252-5c365538e927 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:43.954945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-05b3c82d-d8d5-49f0-b252-5c365538e927 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:43.960834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05b3c82d-d8d5-49f0-b252-5c365538e927 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:43.961325 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05b3c82d-d8d5-49f0-b252-5c365538e927 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:43.961756 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05b3c82d-d8d5-49f0-b252-5c365538e927 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:43.962468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05b3c82d-d8d5-49f0-b252-5c365538e927 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:43.962918 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05b3c82d-d8d5-49f0-b252-5c365538e927 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:43.963338 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05b3c82d-d8d5-49f0-b252-5c365538e927 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:43.967165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:44.028313 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-05b3c82d-d8d5-49f0-b252-5c365538e927 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:44.036138 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-05b3c82d-d8d5-49f0-b252-5c365538e927 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.085663 Sep 08 14:01:44.036695 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 626/1145] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:43 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:44.047418 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0a244145-6717-45cc-98b8-75c773dd228d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:44.049656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0a244145-6717-45cc-98b8-75c773dd228d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:44.056161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0a244145-6717-45cc-98b8-75c773dd228d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:44.056636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0a244145-6717-45cc-98b8-75c773dd228d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:44.057068 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0a244145-6717-45cc-98b8-75c773dd228d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:44.057749 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0a244145-6717-45cc-98b8-75c773dd228d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:44.058231 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0a244145-6717-45cc-98b8-75c773dd228d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:44.058654 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0a244145-6717-45cc-98b8-75c773dd228d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:44.089755 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0a244145-6717-45cc-98b8-75c773dd228d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046291 Sep 08 14:01:44.090422 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 520/1146] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:44 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:44.133496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b548e3d1-6490-44eb-af77-2576b7d11303 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:44.135577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b548e3d1-6490-44eb-af77-2576b7d11303 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:44.143604 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b548e3d1-6490-44eb-af77-2576b7d11303 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:44.144242 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b548e3d1-6490-44eb-af77-2576b7d11303 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:44.144686 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b548e3d1-6490-44eb-af77-2576b7d11303 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:44.145465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b548e3d1-6490-44eb-af77-2576b7d11303 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:44.146252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b548e3d1-6490-44eb-af77-2576b7d11303 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:44.147030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b548e3d1-6490-44eb-af77-2576b7d11303 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:44.178443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b548e3d1-6490-44eb-af77-2576b7d11303 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049254 Sep 08 14:01:44.179008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 627/1147] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:44 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:44.561891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0be54379-a581-4f5b-b980-b02646b42b0d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:44.564148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0be54379-a581-4f5b-b980-b02646b42b0d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:44.570781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0be54379-a581-4f5b-b980-b02646b42b0d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:44.571267 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0be54379-a581-4f5b-b980-b02646b42b0d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:44.571703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0be54379-a581-4f5b-b980-b02646b42b0d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:44.572412 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0be54379-a581-4f5b-b980-b02646b42b0d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:44.572865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0be54379-a581-4f5b-b980-b02646b42b0d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:44.573293 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0be54379-a581-4f5b-b980-b02646b42b0d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:44.604363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0be54379-a581-4f5b-b980-b02646b42b0d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046500 Sep 08 14:01:44.605217 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 521/1148] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:44 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:45.065978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d5802454-70cf-4fdb-9007-170a101a00e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:45.066656 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d5802454-70cf-4fdb-9007-170a101a00e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:45.074531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d5802454-70cf-4fdb-9007-170a101a00e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:45.075021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d5802454-70cf-4fdb-9007-170a101a00e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:45.075447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d5802454-70cf-4fdb-9007-170a101a00e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:45.076146 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d5802454-70cf-4fdb-9007-170a101a00e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:45.076595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d5802454-70cf-4fdb-9007-170a101a00e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:45.077019 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d5802454-70cf-4fdb-9007-170a101a00e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:45.101490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c3d6a02f-ebd5-404c-87e2-9918f1866845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:45.103679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c3d6a02f-ebd5-404c-87e2-9918f1866845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:45.112578 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c3d6a02f-ebd5-404c-87e2-9918f1866845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:45.113065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c3d6a02f-ebd5-404c-87e2-9918f1866845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:45.113496 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c3d6a02f-ebd5-404c-87e2-9918f1866845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:45.114216 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c3d6a02f-ebd5-404c-87e2-9918f1866845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:45.114672 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c3d6a02f-ebd5-404c-87e2-9918f1866845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:45.115080 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c3d6a02f-ebd5-404c-87e2-9918f1866845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:45.140621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d5802454-70cf-4fdb-9007-170a101a00e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:45.149873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c3d6a02f-ebd5-404c-87e2-9918f1866845 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052260 Sep 08 14:01:45.150336 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d5802454-70cf-4fdb-9007-170a101a00e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.094131 Sep 08 14:01:45.150464 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 522/1149] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:45 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:45.150911 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 628/1150] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:45 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:45.191604 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4157a1ea-cf8c-49ca-8322-513535a6bb30 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:45.193766 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4157a1ea-cf8c-49ca-8322-513535a6bb30 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:45.199831 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4157a1ea-cf8c-49ca-8322-513535a6bb30 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:45.200322 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4157a1ea-cf8c-49ca-8322-513535a6bb30 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:45.200762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4157a1ea-cf8c-49ca-8322-513535a6bb30 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:45.201457 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4157a1ea-cf8c-49ca-8322-513535a6bb30 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:45.201910 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4157a1ea-cf8c-49ca-8322-513535a6bb30 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:45.202370 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4157a1ea-cf8c-49ca-8322-513535a6bb30 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:45.219786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:45.233166 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4157a1ea-cf8c-49ca-8322-513535a6bb30 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045463 Sep 08 14:01:45.237173 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 523/1151] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:45 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:45.617298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1db3e238-2535-43ad-bcda-96d629529c71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:45.619529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1db3e238-2535-43ad-bcda-96d629529c71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:45.625680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1db3e238-2535-43ad-bcda-96d629529c71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:45.626201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1db3e238-2535-43ad-bcda-96d629529c71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:45.626645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1db3e238-2535-43ad-bcda-96d629529c71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:45.627355 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1db3e238-2535-43ad-bcda-96d629529c71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:45.627812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1db3e238-2535-43ad-bcda-96d629529c71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:45.628237 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1db3e238-2535-43ad-bcda-96d629529c71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:45.659925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1db3e238-2535-43ad-bcda-96d629529c71 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046685 Sep 08 14:01:45.660487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 629/1152] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:45 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:46.162867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ac264314-6972-4cc2-b496-5d533cec70d6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:46.165086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ac264314-6972-4cc2-b496-5d533cec70d6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:46.172704 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ac264314-6972-4cc2-b496-5d533cec70d6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:46.173192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ac264314-6972-4cc2-b496-5d533cec70d6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:46.173933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ac264314-6972-4cc2-b496-5d533cec70d6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:46.174429 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ac264314-6972-4cc2-b496-5d533cec70d6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:46.174902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ac264314-6972-4cc2-b496-5d533cec70d6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:46.175319 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ac264314-6972-4cc2-b496-5d533cec70d6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:46.176996 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0841d781-18c7-4287-923d-af947738f050 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:46.177605 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0841d781-18c7-4287-923d-af947738f050 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:46.184096 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0841d781-18c7-4287-923d-af947738f050 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:46.184584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0841d781-18c7-4287-923d-af947738f050 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:46.185017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0841d781-18c7-4287-923d-af947738f050 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:46.185712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0841d781-18c7-4287-923d-af947738f050 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:46.186175 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0841d781-18c7-4287-923d-af947738f050 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:46.186601 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0841d781-18c7-4287-923d-af947738f050 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:46.207067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ac264314-6972-4cc2-b496-5d533cec70d6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048194 Sep 08 14:01:46.207656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 524/1153] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:46 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:46.248542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-73675f6d-c104-49b2-8e1a-61e00de45776 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:46.250943 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-73675f6d-c104-49b2-8e1a-61e00de45776 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:46.256462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0841d781-18c7-4287-923d-af947738f050 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:46.258699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73675f6d-c104-49b2-8e1a-61e00de45776 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:46.259198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73675f6d-c104-49b2-8e1a-61e00de45776 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:46.259634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73675f6d-c104-49b2-8e1a-61e00de45776 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:46.260387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73675f6d-c104-49b2-8e1a-61e00de45776 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:46.260810 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73675f6d-c104-49b2-8e1a-61e00de45776 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:46.261231 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73675f6d-c104-49b2-8e1a-61e00de45776 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:46.267617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0841d781-18c7-4287-923d-af947738f050 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.097434 Sep 08 14:01:46.268565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 630/1154] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:46 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:01:46.296330 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-73675f6d-c104-49b2-8e1a-61e00de45776 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051757 Sep 08 14:01:46.296816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 525/1155] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:46 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:46.673457 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-77fdede6-2295-4ef4-b8e4-ccced38ca29f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:46.675704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-77fdede6-2295-4ef4-b8e4-ccced38ca29f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:46.682288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77fdede6-2295-4ef4-b8e4-ccced38ca29f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:46.682784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77fdede6-2295-4ef4-b8e4-ccced38ca29f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:46.683227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77fdede6-2295-4ef4-b8e4-ccced38ca29f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:46.683926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77fdede6-2295-4ef4-b8e4-ccced38ca29f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:46.684385 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77fdede6-2295-4ef4-b8e4-ccced38ca29f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:46.684815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77fdede6-2295-4ef4-b8e4-ccced38ca29f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:46.715522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-77fdede6-2295-4ef4-b8e4-ccced38ca29f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046149 Sep 08 14:01:46.716091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 631/1156] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:46 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:47.220411 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d5a4a2b8-25b5-4387-952b-09ce8dccbd4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:47.222542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d5a4a2b8-25b5-4387-952b-09ce8dccbd4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:47.229833 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d5a4a2b8-25b5-4387-952b-09ce8dccbd4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:47.230343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d5a4a2b8-25b5-4387-952b-09ce8dccbd4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:47.230773 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d5a4a2b8-25b5-4387-952b-09ce8dccbd4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:47.231451 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d5a4a2b8-25b5-4387-952b-09ce8dccbd4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:47.231949 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d5a4a2b8-25b5-4387-952b-09ce8dccbd4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:47.232335 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d5a4a2b8-25b5-4387-952b-09ce8dccbd4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:47.263347 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d5a4a2b8-25b5-4387-952b-09ce8dccbd4e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046905 Sep 08 14:01:47.263917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 526/1157] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:47 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:47.291335 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-39c6a03a-cb24-434a-94d3-3529598a6a34 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:47.291937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-39c6a03a-cb24-434a-94d3-3529598a6a34 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:47.298368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-39c6a03a-cb24-434a-94d3-3529598a6a34 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:47.298838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-39c6a03a-cb24-434a-94d3-3529598a6a34 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:47.299273 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-39c6a03a-cb24-434a-94d3-3529598a6a34 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:47.299962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-39c6a03a-cb24-434a-94d3-3529598a6a34 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:47.300408 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-39c6a03a-cb24-434a-94d3-3529598a6a34 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:47.300834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-39c6a03a-cb24-434a-94d3-3529598a6a34 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:47.309077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-60558e13-9255-4c4e-afba-8cd51303bcb0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:47.311772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-60558e13-9255-4c4e-afba-8cd51303bcb0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:47.317877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-60558e13-9255-4c4e-afba-8cd51303bcb0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:47.318403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-60558e13-9255-4c4e-afba-8cd51303bcb0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:47.318844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-60558e13-9255-4c4e-afba-8cd51303bcb0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:47.319537 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-60558e13-9255-4c4e-afba-8cd51303bcb0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:47.319987 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-60558e13-9255-4c4e-afba-8cd51303bcb0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:47.320414 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-60558e13-9255-4c4e-afba-8cd51303bcb0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:47.351022 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-60558e13-9255-4c4e-afba-8cd51303bcb0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046599 Sep 08 14:01:47.352123 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 527/1158] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:47 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:47.365093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-39c6a03a-cb24-434a-94d3-3529598a6a34 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:47.373362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-39c6a03a-cb24-434a-94d3-3529598a6a34 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.086294 Sep 08 14:01:47.373932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 632/1159] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:47 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:47.728988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-df7bbfa9-e569-4cb3-b3dc-44a863faf40d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:47.731224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-df7bbfa9-e569-4cb3-b3dc-44a863faf40d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:47.737364 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-df7bbfa9-e569-4cb3-b3dc-44a863faf40d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:47.737855 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-df7bbfa9-e569-4cb3-b3dc-44a863faf40d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:47.738326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-df7bbfa9-e569-4cb3-b3dc-44a863faf40d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:47.739032 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-df7bbfa9-e569-4cb3-b3dc-44a863faf40d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:47.739489 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-df7bbfa9-e569-4cb3-b3dc-44a863faf40d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:47.739913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-df7bbfa9-e569-4cb3-b3dc-44a863faf40d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:47.771154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-df7bbfa9-e569-4cb3-b3dc-44a863faf40d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046715 Sep 08 14:01:47.771743 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 528/1160] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:47 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:48.275840 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2c961141-2647-42b0-a872-5f83d422fbce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:48.278090 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2c961141-2647-42b0-a872-5f83d422fbce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:48.284591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c961141-2647-42b0-a872-5f83d422fbce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:48.285079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c961141-2647-42b0-a872-5f83d422fbce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:48.285517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c961141-2647-42b0-a872-5f83d422fbce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:48.286249 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c961141-2647-42b0-a872-5f83d422fbce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:48.286708 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c961141-2647-42b0-a872-5f83d422fbce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:48.287132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c961141-2647-42b0-a872-5f83d422fbce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:48.322814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2c961141-2647-42b0-a872-5f83d422fbce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050230 Sep 08 14:01:48.323588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 633/1161] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:48 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:48.364352 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3d0de4d2-9d94-4759-97bd-1f5eb022e22e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:48.366572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3d0de4d2-9d94-4759-97bd-1f5eb022e22e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:48.373647 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d0de4d2-9d94-4759-97bd-1f5eb022e22e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:48.374126 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d0de4d2-9d94-4759-97bd-1f5eb022e22e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:48.374587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d0de4d2-9d94-4759-97bd-1f5eb022e22e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:48.375276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d0de4d2-9d94-4759-97bd-1f5eb022e22e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:48.375716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d0de4d2-9d94-4759-97bd-1f5eb022e22e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:48.376130 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d0de4d2-9d94-4759-97bd-1f5eb022e22e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:48.396446 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fb520dda-04c1-47b2-9a3d-01c9f47c2d7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:48.397181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fb520dda-04c1-47b2-9a3d-01c9f47c2d7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:48.403389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb520dda-04c1-47b2-9a3d-01c9f47c2d7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:48.403883 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb520dda-04c1-47b2-9a3d-01c9f47c2d7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:48.404393 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb520dda-04c1-47b2-9a3d-01c9f47c2d7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:48.405102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb520dda-04c1-47b2-9a3d-01c9f47c2d7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:48.405555 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb520dda-04c1-47b2-9a3d-01c9f47c2d7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:48.405979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb520dda-04c1-47b2-9a3d-01c9f47c2d7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:48.406879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3d0de4d2-9d94-4759-97bd-1f5eb022e22e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046505 Sep 08 14:01:48.407463 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 529/1162] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:48 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:48.483422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fb520dda-04c1-47b2-9a3d-01c9f47c2d7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:48.493196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fb520dda-04c1-47b2-9a3d-01c9f47c2d7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.099829 Sep 08 14:01:48.493964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 634/1163] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:48 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:48.784860 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6180ab1b-ada3-493a-8aaf-bb4b453e15aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:48.786970 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6180ab1b-ada3-493a-8aaf-bb4b453e15aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:48.793531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6180ab1b-ada3-493a-8aaf-bb4b453e15aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:48.794016 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6180ab1b-ada3-493a-8aaf-bb4b453e15aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:48.794479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6180ab1b-ada3-493a-8aaf-bb4b453e15aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:48.795353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6180ab1b-ada3-493a-8aaf-bb4b453e15aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:48.795807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6180ab1b-ada3-493a-8aaf-bb4b453e15aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:48.796232 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6180ab1b-ada3-493a-8aaf-bb4b453e15aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:48.827606 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6180ab1b-ada3-493a-8aaf-bb4b453e15aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046798 Sep 08 14:01:48.828187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 530/1164] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:48 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:49.335645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-74883e49-d67f-4fde-8c2f-bd530d14e612 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:49.338466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-74883e49-d67f-4fde-8c2f-bd530d14e612 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:49.345474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-74883e49-d67f-4fde-8c2f-bd530d14e612 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:49.345965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-74883e49-d67f-4fde-8c2f-bd530d14e612 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:49.346434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-74883e49-d67f-4fde-8c2f-bd530d14e612 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:49.347139 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-74883e49-d67f-4fde-8c2f-bd530d14e612 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:49.347594 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-74883e49-d67f-4fde-8c2f-bd530d14e612 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:49.348021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-74883e49-d67f-4fde-8c2f-bd530d14e612 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:49.378405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-74883e49-d67f-4fde-8c2f-bd530d14e612 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046726 Sep 08 14:01:49.378970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 635/1165] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:49 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:49.420531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-519a6c7a-a727-4c49-98c0-ba6e117e6a72 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:49.422707 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-519a6c7a-a727-4c49-98c0-ba6e117e6a72 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:49.429117 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-519a6c7a-a727-4c49-98c0-ba6e117e6a72 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:49.429604 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-519a6c7a-a727-4c49-98c0-ba6e117e6a72 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:49.430049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-519a6c7a-a727-4c49-98c0-ba6e117e6a72 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:49.431351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-519a6c7a-a727-4c49-98c0-ba6e117e6a72 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:49.431805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-519a6c7a-a727-4c49-98c0-ba6e117e6a72 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:49.432229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-519a6c7a-a727-4c49-98c0-ba6e117e6a72 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:49.462636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-519a6c7a-a727-4c49-98c0-ba6e117e6a72 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046064 Sep 08 14:01:49.463213 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 531/1166] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:49 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:49.518039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-400b63f9-3b09-40b1-a8b8-11fbc26f20fa tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:49.518679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-400b63f9-3b09-40b1-a8b8-11fbc26f20fa tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:49.524800 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-400b63f9-3b09-40b1-a8b8-11fbc26f20fa tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:49.525284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-400b63f9-3b09-40b1-a8b8-11fbc26f20fa tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:49.525728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-400b63f9-3b09-40b1-a8b8-11fbc26f20fa tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:49.526454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-400b63f9-3b09-40b1-a8b8-11fbc26f20fa tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:49.526912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-400b63f9-3b09-40b1-a8b8-11fbc26f20fa tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:49.527334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-400b63f9-3b09-40b1-a8b8-11fbc26f20fa tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:49.591179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-400b63f9-3b09-40b1-a8b8-11fbc26f20fa tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:49.599463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-400b63f9-3b09-40b1-a8b8-11fbc26f20fa tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.085346 Sep 08 14:01:49.600024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 636/1167] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:49 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:49.840256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3503a49e-b1ac-435b-ba80-f2c1860b402a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:49.842339 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3503a49e-b1ac-435b-ba80-f2c1860b402a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:49.848449 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3503a49e-b1ac-435b-ba80-f2c1860b402a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:49.848936 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3503a49e-b1ac-435b-ba80-f2c1860b402a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:49.849372 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3503a49e-b1ac-435b-ba80-f2c1860b402a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:49.850070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3503a49e-b1ac-435b-ba80-f2c1860b402a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:49.850582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3503a49e-b1ac-435b-ba80-f2c1860b402a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:49.851010 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3503a49e-b1ac-435b-ba80-f2c1860b402a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:49.884039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3503a49e-b1ac-435b-ba80-f2c1860b402a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048355 Sep 08 14:01:49.884626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 532/1168] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:49 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:50.390948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-28e90136-fd9c-48d0-af0a-0976d054ae41 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:50.393244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-28e90136-fd9c-48d0-af0a-0976d054ae41 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:50.399836 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-28e90136-fd9c-48d0-af0a-0976d054ae41 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:50.400315 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-28e90136-fd9c-48d0-af0a-0976d054ae41 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:50.400748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-28e90136-fd9c-48d0-af0a-0976d054ae41 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:50.401446 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-28e90136-fd9c-48d0-af0a-0976d054ae41 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:50.402035 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-28e90136-fd9c-48d0-af0a-0976d054ae41 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:50.402581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-28e90136-fd9c-48d0-af0a-0976d054ae41 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:50.433652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-28e90136-fd9c-48d0-af0a-0976d054ae41 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046605 Sep 08 14:01:50.434235 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 637/1169] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:50 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:50.474270 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6af80991-3d54-4088-9ff8-e8245e6f415f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:50.476402 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6af80991-3d54-4088-9ff8-e8245e6f415f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:50.483282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6af80991-3d54-4088-9ff8-e8245e6f415f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:50.483762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6af80991-3d54-4088-9ff8-e8245e6f415f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:50.484202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6af80991-3d54-4088-9ff8-e8245e6f415f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:50.484898 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6af80991-3d54-4088-9ff8-e8245e6f415f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:50.485353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6af80991-3d54-4088-9ff8-e8245e6f415f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:50.485781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6af80991-3d54-4088-9ff8-e8245e6f415f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:50.540465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6af80991-3d54-4088-9ff8-e8245e6f415f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069380 Sep 08 14:01:50.541065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 533/1170] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:50 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:50.623551 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b77173ba-6b1c-488b-9cf8-9739a0d3a476 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:50.624442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b77173ba-6b1c-488b-9cf8-9739a0d3a476 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:50.631902 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b77173ba-6b1c-488b-9cf8-9739a0d3a476 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:50.632410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b77173ba-6b1c-488b-9cf8-9739a0d3a476 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:50.633081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b77173ba-6b1c-488b-9cf8-9739a0d3a476 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:50.634811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b77173ba-6b1c-488b-9cf8-9739a0d3a476 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:50.635399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b77173ba-6b1c-488b-9cf8-9739a0d3a476 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:50.635844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b77173ba-6b1c-488b-9cf8-9739a0d3a476 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:50.704590 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b77173ba-6b1c-488b-9cf8-9739a0d3a476 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:50.712748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b77173ba-6b1c-488b-9cf8-9739a0d3a476 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.093067 Sep 08 14:01:50.713322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 638/1171] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:50 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:50.896854 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-28c6cd53-9848-49a4-a014-17712ca7863f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:50.899021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-28c6cd53-9848-49a4-a014-17712ca7863f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:50.905202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-28c6cd53-9848-49a4-a014-17712ca7863f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:50.905722 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-28c6cd53-9848-49a4-a014-17712ca7863f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:50.906181 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-28c6cd53-9848-49a4-a014-17712ca7863f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:50.906874 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-28c6cd53-9848-49a4-a014-17712ca7863f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:50.907323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-28c6cd53-9848-49a4-a014-17712ca7863f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:50.907747 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-28c6cd53-9848-49a4-a014-17712ca7863f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:50.945137 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-28c6cd53-9848-49a4-a014-17712ca7863f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052226 Sep 08 14:01:50.945729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 534/1172] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:50 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:51.446656 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-407a7f4f-c6d4-46fc-809d-26c8c1243d79 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:51.448803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-407a7f4f-c6d4-46fc-809d-26c8c1243d79 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:51.455436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-407a7f4f-c6d4-46fc-809d-26c8c1243d79 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:51.455930 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-407a7f4f-c6d4-46fc-809d-26c8c1243d79 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:51.456374 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-407a7f4f-c6d4-46fc-809d-26c8c1243d79 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:51.457068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-407a7f4f-c6d4-46fc-809d-26c8c1243d79 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:51.457521 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-407a7f4f-c6d4-46fc-809d-26c8c1243d79 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:51.457950 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-407a7f4f-c6d4-46fc-809d-26c8c1243d79 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:51.489106 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-407a7f4f-c6d4-46fc-809d-26c8c1243d79 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046458 Sep 08 14:01:51.489665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 639/1173] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:51 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:51.555758 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0dcc97dd-ef3c-4f4a-a4b4-b4881e966e7a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:51.557965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0dcc97dd-ef3c-4f4a-a4b4-b4881e966e7a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:51.564584 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0dcc97dd-ef3c-4f4a-a4b4-b4881e966e7a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:51.565065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0dcc97dd-ef3c-4f4a-a4b4-b4881e966e7a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:51.565497 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0dcc97dd-ef3c-4f4a-a4b4-b4881e966e7a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:51.566212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0dcc97dd-ef3c-4f4a-a4b4-b4881e966e7a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:51.566661 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0dcc97dd-ef3c-4f4a-a4b4-b4881e966e7a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:51.567083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0dcc97dd-ef3c-4f4a-a4b4-b4881e966e7a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:51.599949 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0dcc97dd-ef3c-4f4a-a4b4-b4881e966e7a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048027 Sep 08 14:01:51.600523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 535/1174] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:51 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:51.737206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-670625a9-bb33-4f29-8467-23e5bd70f07e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:51.737711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-670625a9-bb33-4f29-8467-23e5bd70f07e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:51.739604 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:51.744218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-670625a9-bb33-4f29-8467-23e5bd70f07e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:51.744703 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-670625a9-bb33-4f29-8467-23e5bd70f07e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:51.745146 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-670625a9-bb33-4f29-8467-23e5bd70f07e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:51.745836 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-670625a9-bb33-4f29-8467-23e5bd70f07e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:51.746328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-670625a9-bb33-4f29-8467-23e5bd70f07e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:51.746755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-670625a9-bb33-4f29-8467-23e5bd70f07e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:51.807800 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-670625a9-bb33-4f29-8467-23e5bd70f07e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:51.815886 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-670625a9-bb33-4f29-8467-23e5bd70f07e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.083585 Sep 08 14:01:51.816479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 640/1175] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:51 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:51.957104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0fe94d37-2fa5-4a39-80da-6784e97733e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:51.959272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0fe94d37-2fa5-4a39-80da-6784e97733e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:51.965973 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0fe94d37-2fa5-4a39-80da-6784e97733e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:51.966521 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0fe94d37-2fa5-4a39-80da-6784e97733e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:51.967096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0fe94d37-2fa5-4a39-80da-6784e97733e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:51.967791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0fe94d37-2fa5-4a39-80da-6784e97733e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:51.968241 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0fe94d37-2fa5-4a39-80da-6784e97733e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:51.968686 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0fe94d37-2fa5-4a39-80da-6784e97733e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:52.003345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0fe94d37-2fa5-4a39-80da-6784e97733e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049777 Sep 08 14:01:52.004272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 536/1176] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:51 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:52.501225 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0da6b6c4-28e8-4436-ae97-b858a51ceb8b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:52.517902 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0da6b6c4-28e8-4436-ae97-b858a51ceb8b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:52.517902 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0da6b6c4-28e8-4436-ae97-b858a51ceb8b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:52.517902 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0da6b6c4-28e8-4436-ae97-b858a51ceb8b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:52.517902 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0da6b6c4-28e8-4436-ae97-b858a51ceb8b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:52.518669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0da6b6c4-28e8-4436-ae97-b858a51ceb8b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:52.518669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0da6b6c4-28e8-4436-ae97-b858a51ceb8b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:52.518669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0da6b6c4-28e8-4436-ae97-b858a51ceb8b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:52.544852 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0da6b6c4-28e8-4436-ae97-b858a51ceb8b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047487 Sep 08 14:01:52.545309 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 641/1177] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:52 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:52.616061 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e8dbdac3-cf10-4c31-a3e4-2747ee3ea1c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:52.618782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e8dbdac3-cf10-4c31-a3e4-2747ee3ea1c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:52.626952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8dbdac3-cf10-4c31-a3e4-2747ee3ea1c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:52.627439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8dbdac3-cf10-4c31-a3e4-2747ee3ea1c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:52.627877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8dbdac3-cf10-4c31-a3e4-2747ee3ea1c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:52.628584 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8dbdac3-cf10-4c31-a3e4-2747ee3ea1c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:52.629047 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8dbdac3-cf10-4c31-a3e4-2747ee3ea1c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:52.629472 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8dbdac3-cf10-4c31-a3e4-2747ee3ea1c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:52.660776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e8dbdac3-cf10-4c31-a3e4-2747ee3ea1c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048646 Sep 08 14:01:52.661360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 537/1178] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:52 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:52.839383 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3f8534e9-de9c-4235-922a-bdbf3ff3b333 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:52.839982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3f8534e9-de9c-4235-922a-bdbf3ff3b333 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:52.845815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3f8534e9-de9c-4235-922a-bdbf3ff3b333 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:52.846334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3f8534e9-de9c-4235-922a-bdbf3ff3b333 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:52.846772 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3f8534e9-de9c-4235-922a-bdbf3ff3b333 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:52.847459 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3f8534e9-de9c-4235-922a-bdbf3ff3b333 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:52.847908 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3f8534e9-de9c-4235-922a-bdbf3ff3b333 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:52.848322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3f8534e9-de9c-4235-922a-bdbf3ff3b333 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:52.913155 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3f8534e9-de9c-4235-922a-bdbf3ff3b333 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:52.915410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:52.921713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3f8534e9-de9c-4235-922a-bdbf3ff3b333 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.086148 Sep 08 14:01:52.922320 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 642/1179] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:52 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:53.016021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-39a24059-3e98-41ee-8ed5-76efb23efe19 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:53.018226 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-39a24059-3e98-41ee-8ed5-76efb23efe19 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:53.030514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39a24059-3e98-41ee-8ed5-76efb23efe19 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:53.030913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39a24059-3e98-41ee-8ed5-76efb23efe19 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:53.031352 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39a24059-3e98-41ee-8ed5-76efb23efe19 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:53.032078 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39a24059-3e98-41ee-8ed5-76efb23efe19 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:53.032515 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39a24059-3e98-41ee-8ed5-76efb23efe19 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:53.032933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39a24059-3e98-41ee-8ed5-76efb23efe19 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:53.064642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-39a24059-3e98-41ee-8ed5-76efb23efe19 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052573 Sep 08 14:01:53.065212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 538/1180] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:53 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:53.557627 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9cf9b813-587a-42ad-8a2c-58984379cf93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:53.559906 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9cf9b813-587a-42ad-8a2c-58984379cf93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:53.566781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9cf9b813-587a-42ad-8a2c-58984379cf93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:53.567270 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9cf9b813-587a-42ad-8a2c-58984379cf93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:53.567710 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9cf9b813-587a-42ad-8a2c-58984379cf93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:53.568402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9cf9b813-587a-42ad-8a2c-58984379cf93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:53.568862 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9cf9b813-587a-42ad-8a2c-58984379cf93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:53.569291 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9cf9b813-587a-42ad-8a2c-58984379cf93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:53.599402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9cf9b813-587a-42ad-8a2c-58984379cf93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046019 Sep 08 14:01:53.599966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 643/1181] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:53 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:53.672480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0c43cb36-4070-4773-9424-fd7e6f914472 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:53.674682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0c43cb36-4070-4773-9424-fd7e6f914472 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:53.681501 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0c43cb36-4070-4773-9424-fd7e6f914472 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:53.681989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0c43cb36-4070-4773-9424-fd7e6f914472 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:53.682450 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0c43cb36-4070-4773-9424-fd7e6f914472 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:53.683154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0c43cb36-4070-4773-9424-fd7e6f914472 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:53.683607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0c43cb36-4070-4773-9424-fd7e6f914472 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:53.684148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0c43cb36-4070-4773-9424-fd7e6f914472 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:53.718829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0c43cb36-4070-4773-9424-fd7e6f914472 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050314 Sep 08 14:01:53.719400 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 539/1182] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:53 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:53.945102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d8e8a7c9-3e76-4d7e-aca2-5eeb88f01014 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:53.946274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d8e8a7c9-3e76-4d7e-aca2-5eeb88f01014 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:53.952464 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d8e8a7c9-3e76-4d7e-aca2-5eeb88f01014 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:53.952956 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d8e8a7c9-3e76-4d7e-aca2-5eeb88f01014 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:53.953403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d8e8a7c9-3e76-4d7e-aca2-5eeb88f01014 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:53.954108 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d8e8a7c9-3e76-4d7e-aca2-5eeb88f01014 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:53.954601 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d8e8a7c9-3e76-4d7e-aca2-5eeb88f01014 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:53.955030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d8e8a7c9-3e76-4d7e-aca2-5eeb88f01014 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:54.019070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d8e8a7c9-3e76-4d7e-aca2-5eeb88f01014 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:54.027642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d8e8a7c9-3e76-4d7e-aca2-5eeb88f01014 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.086411 Sep 08 14:01:54.028212 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 644/1183] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:53 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:54.077749 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-672c921f-2597-4da8-aa30-179bf6112080 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:54.079920 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-672c921f-2597-4da8-aa30-179bf6112080 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:54.086777 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-672c921f-2597-4da8-aa30-179bf6112080 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:54.087266 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-672c921f-2597-4da8-aa30-179bf6112080 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:54.087699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-672c921f-2597-4da8-aa30-179bf6112080 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:54.088402 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-672c921f-2597-4da8-aa30-179bf6112080 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:54.088855 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-672c921f-2597-4da8-aa30-179bf6112080 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:54.089280 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-672c921f-2597-4da8-aa30-179bf6112080 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:54.093279 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:54.121099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-672c921f-2597-4da8-aa30-179bf6112080 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047271 Sep 08 14:01:54.121570 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 540/1184] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:54 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:54.613201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1899c9c9-2129-4392-82f3-5e06d7030624 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:54.615390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1899c9c9-2129-4392-82f3-5e06d7030624 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:54.622000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1899c9c9-2129-4392-82f3-5e06d7030624 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:54.622528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1899c9c9-2129-4392-82f3-5e06d7030624 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:54.622970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1899c9c9-2129-4392-82f3-5e06d7030624 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:54.623678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1899c9c9-2129-4392-82f3-5e06d7030624 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:54.624136 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1899c9c9-2129-4392-82f3-5e06d7030624 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:54.624564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1899c9c9-2129-4392-82f3-5e06d7030624 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:54.657171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1899c9c9-2129-4392-82f3-5e06d7030624 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048406 Sep 08 14:01:54.657722 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 645/1185] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:54 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:54.732387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5d62dd48-4e64-47d7-b28e-903ec865e505 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:54.734683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5d62dd48-4e64-47d7-b28e-903ec865e505 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:54.741785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d62dd48-4e64-47d7-b28e-903ec865e505 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:54.742969 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d62dd48-4e64-47d7-b28e-903ec865e505 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:54.743407 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d62dd48-4e64-47d7-b28e-903ec865e505 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:54.744099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d62dd48-4e64-47d7-b28e-903ec865e505 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:54.744542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d62dd48-4e64-47d7-b28e-903ec865e505 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:54.744963 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d62dd48-4e64-47d7-b28e-903ec865e505 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:54.775652 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5d62dd48-4e64-47d7-b28e-903ec865e505 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047618 Sep 08 14:01:54.776251 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 541/1186] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:54 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:55.050743 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3be1d4bd-0259-4c03-bbbb-ded17978c5f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:55.051274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3be1d4bd-0259-4c03-bbbb-ded17978c5f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:55.057502 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3be1d4bd-0259-4c03-bbbb-ded17978c5f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:55.058002 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3be1d4bd-0259-4c03-bbbb-ded17978c5f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:55.058477 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3be1d4bd-0259-4c03-bbbb-ded17978c5f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:55.059182 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3be1d4bd-0259-4c03-bbbb-ded17978c5f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:55.059641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3be1d4bd-0259-4c03-bbbb-ded17978c5f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:55.060070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3be1d4bd-0259-4c03-bbbb-ded17978c5f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:55.123551 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3be1d4bd-0259-4c03-bbbb-ded17978c5f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:55.132209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3be1d4bd-0259-4c03-bbbb-ded17978c5f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.085356 Sep 08 14:01:55.133013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 646/1187] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:55 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:55.133840 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6df1f5fd-8119-4264-a50c-435687750725 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:55.136337 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6df1f5fd-8119-4264-a50c-435687750725 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:55.142974 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6df1f5fd-8119-4264-a50c-435687750725 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:55.143456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6df1f5fd-8119-4264-a50c-435687750725 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:55.143884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6df1f5fd-8119-4264-a50c-435687750725 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:55.144572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6df1f5fd-8119-4264-a50c-435687750725 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:55.145016 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6df1f5fd-8119-4264-a50c-435687750725 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:55.145438 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6df1f5fd-8119-4264-a50c-435687750725 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:55.176177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6df1f5fd-8119-4264-a50c-435687750725 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046977 Sep 08 14:01:55.176727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 542/1188] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:55 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:01:55.673073 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-597ce4ef-5300-4d3c-8262-18cc45fdf8a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:55.675299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-597ce4ef-5300-4d3c-8262-18cc45fdf8a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:55.682365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-597ce4ef-5300-4d3c-8262-18cc45fdf8a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:55.682849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-597ce4ef-5300-4d3c-8262-18cc45fdf8a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:55.683286 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-597ce4ef-5300-4d3c-8262-18cc45fdf8a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:55.683978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-597ce4ef-5300-4d3c-8262-18cc45fdf8a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:55.684439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-597ce4ef-5300-4d3c-8262-18cc45fdf8a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:55.684869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-597ce4ef-5300-4d3c-8262-18cc45fdf8a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:55.715746 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-597ce4ef-5300-4d3c-8262-18cc45fdf8a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046626 Sep 08 14:01:55.716319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 647/1189] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:55 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:55.789573 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-dd2ae913-ac19-41c2-a037-bd64d5c3ed13 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:55.791716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-dd2ae913-ac19-41c2-a037-bd64d5c3ed13 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:55.798365 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd2ae913-ac19-41c2-a037-bd64d5c3ed13 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:55.798843 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd2ae913-ac19-41c2-a037-bd64d5c3ed13 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:55.799271 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd2ae913-ac19-41c2-a037-bd64d5c3ed13 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:55.799958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd2ae913-ac19-41c2-a037-bd64d5c3ed13 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:55.800406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd2ae913-ac19-41c2-a037-bd64d5c3ed13 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:55.800825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dd2ae913-ac19-41c2-a037-bd64d5c3ed13 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:55.830439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-dd2ae913-ac19-41c2-a037-bd64d5c3ed13 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044900 Sep 08 14:01:55.830996 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 543/1190] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:55 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:56.155214 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-916f9a37-bcb7-4e11-be6d-b0837a381d2a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:56.155859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-916f9a37-bcb7-4e11-be6d-b0837a381d2a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:56.161960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-916f9a37-bcb7-4e11-be6d-b0837a381d2a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:56.162491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-916f9a37-bcb7-4e11-be6d-b0837a381d2a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:56.162937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-916f9a37-bcb7-4e11-be6d-b0837a381d2a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:56.163638 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-916f9a37-bcb7-4e11-be6d-b0837a381d2a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:56.164622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-916f9a37-bcb7-4e11-be6d-b0837a381d2a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:56.165052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-916f9a37-bcb7-4e11-be6d-b0837a381d2a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:56.189025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a6908318-e59e-4ac5-a7df-7054b555d51a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:56.191629 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a6908318-e59e-4ac5-a7df-7054b555d51a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:56.197570 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6908318-e59e-4ac5-a7df-7054b555d51a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:56.198174 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6908318-e59e-4ac5-a7df-7054b555d51a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:56.198614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6908318-e59e-4ac5-a7df-7054b555d51a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:56.199317 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6908318-e59e-4ac5-a7df-7054b555d51a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:56.200108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6908318-e59e-4ac5-a7df-7054b555d51a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:56.200291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6908318-e59e-4ac5-a7df-7054b555d51a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:56.227379 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-916f9a37-bcb7-4e11-be6d-b0837a381d2a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:56.232157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a6908318-e59e-4ac5-a7df-7054b555d51a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047112 Sep 08 14:01:56.232933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 544/1191] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:56 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:01:56.235687 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-916f9a37-bcb7-4e11-be6d-b0837a381d2a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.084412 Sep 08 14:01:56.236280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 648/1192] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:56 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:56.730107 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e8469dbf-0cfc-4f4b-9d63-5a7c3940c5c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:56.732295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e8469dbf-0cfc-4f4b-9d63-5a7c3940c5c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:56.738518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8469dbf-0cfc-4f4b-9d63-5a7c3940c5c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:56.739095 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8469dbf-0cfc-4f4b-9d63-5a7c3940c5c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:56.739541 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8469dbf-0cfc-4f4b-9d63-5a7c3940c5c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:56.740248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8469dbf-0cfc-4f4b-9d63-5a7c3940c5c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:56.740703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8469dbf-0cfc-4f4b-9d63-5a7c3940c5c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:56.741132 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8469dbf-0cfc-4f4b-9d63-5a7c3940c5c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:56.776321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e8469dbf-0cfc-4f4b-9d63-5a7c3940c5c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051339 Sep 08 14:01:56.776897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 545/1193] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:56 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:56.843602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e1df4d45-2b85-40fa-b988-a7c172858812 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:56.846182 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e1df4d45-2b85-40fa-b988-a7c172858812 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:56.853303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1df4d45-2b85-40fa-b988-a7c172858812 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:56.853787 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1df4d45-2b85-40fa-b988-a7c172858812 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:56.854260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1df4d45-2b85-40fa-b988-a7c172858812 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:56.854959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1df4d45-2b85-40fa-b988-a7c172858812 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:56.855419 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1df4d45-2b85-40fa-b988-a7c172858812 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:56.855845 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1df4d45-2b85-40fa-b988-a7c172858812 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:56.899087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e1df4d45-2b85-40fa-b988-a7c172858812 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059932 Sep 08 14:01:56.900673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 649/1194] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:56 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:57.245143 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ca812fca-2217-4e96-8548-6ee41a0648c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:57.247332 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ca812fca-2217-4e96-8548-6ee41a0648c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:57.257592 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca812fca-2217-4e96-8548-6ee41a0648c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:57.257592 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca812fca-2217-4e96-8548-6ee41a0648c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:57.257592 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca812fca-2217-4e96-8548-6ee41a0648c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:57.257592 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca812fca-2217-4e96-8548-6ee41a0648c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:57.258012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca812fca-2217-4e96-8548-6ee41a0648c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:57.258012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca812fca-2217-4e96-8548-6ee41a0648c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:57.268466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-716267d3-1bea-4d71-abbb-dc570fe8ef90 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:57.269262 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-716267d3-1bea-4d71-abbb-dc570fe8ef90 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:57.276658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-716267d3-1bea-4d71-abbb-dc570fe8ef90 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:57.277146 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-716267d3-1bea-4d71-abbb-dc570fe8ef90 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:57.277579 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-716267d3-1bea-4d71-abbb-dc570fe8ef90 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:57.278579 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-716267d3-1bea-4d71-abbb-dc570fe8ef90 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:57.279093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-716267d3-1bea-4d71-abbb-dc570fe8ef90 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:57.279561 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-716267d3-1bea-4d71-abbb-dc570fe8ef90 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:57.303109 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ca812fca-2217-4e96-8548-6ee41a0648c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.062274 Sep 08 14:01:57.303709 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 546/1195] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:57 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:57.360473 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-716267d3-1bea-4d71-abbb-dc570fe8ef90 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:57.368685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-716267d3-1bea-4d71-abbb-dc570fe8ef90 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.104522 Sep 08 14:01:57.369268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 650/1196] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:57 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:01:57.789155 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6d4c0d74-b2d3-4768-9f0a-f6f4943cf9fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:57.791319 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6d4c0d74-b2d3-4768-9f0a-f6f4943cf9fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:57.798638 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d4c0d74-b2d3-4768-9f0a-f6f4943cf9fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:57.799120 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d4c0d74-b2d3-4768-9f0a-f6f4943cf9fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:57.799549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d4c0d74-b2d3-4768-9f0a-f6f4943cf9fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:57.800281 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d4c0d74-b2d3-4768-9f0a-f6f4943cf9fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:57.800759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d4c0d74-b2d3-4768-9f0a-f6f4943cf9fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:57.801178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d4c0d74-b2d3-4768-9f0a-f6f4943cf9fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:57.832574 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6d4c0d74-b2d3-4768-9f0a-f6f4943cf9fd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047386 Sep 08 14:01:57.833151 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 547/1197] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:57 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:57.913375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e1aa136a-eee5-4412-b31a-fde690d9979e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:57.915525 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e1aa136a-eee5-4412-b31a-fde690d9979e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:57.923800 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1aa136a-eee5-4412-b31a-fde690d9979e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:57.924612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1aa136a-eee5-4412-b31a-fde690d9979e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:57.925372 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1aa136a-eee5-4412-b31a-fde690d9979e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:57.926121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1aa136a-eee5-4412-b31a-fde690d9979e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:57.926600 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1aa136a-eee5-4412-b31a-fde690d9979e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:57.927022 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1aa136a-eee5-4412-b31a-fde690d9979e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:57.960919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e1aa136a-eee5-4412-b31a-fde690d9979e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051666 Sep 08 14:01:57.961516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 651/1198] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:57 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:58.315970 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-da1fe27c-37cc-48ea-b238-ac8bf24d7969 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:58.318062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-da1fe27c-37cc-48ea-b238-ac8bf24d7969 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:58.325200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da1fe27c-37cc-48ea-b238-ac8bf24d7969 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:58.325677 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da1fe27c-37cc-48ea-b238-ac8bf24d7969 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:58.326104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da1fe27c-37cc-48ea-b238-ac8bf24d7969 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:58.326819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da1fe27c-37cc-48ea-b238-ac8bf24d7969 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:58.327262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da1fe27c-37cc-48ea-b238-ac8bf24d7969 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:58.327682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da1fe27c-37cc-48ea-b238-ac8bf24d7969 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:58.358166 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-da1fe27c-37cc-48ea-b238-ac8bf24d7969 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046188 Sep 08 14:01:58.358744 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 548/1199] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:58 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:58.391646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-25fd1fbe-a44b-47cd-976c-41051459cd22 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:58.392254 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-25fd1fbe-a44b-47cd-976c-41051459cd22 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:58.398442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25fd1fbe-a44b-47cd-976c-41051459cd22 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:58.398949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25fd1fbe-a44b-47cd-976c-41051459cd22 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:58.399396 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25fd1fbe-a44b-47cd-976c-41051459cd22 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:58.400093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25fd1fbe-a44b-47cd-976c-41051459cd22 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:58.400545 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25fd1fbe-a44b-47cd-976c-41051459cd22 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:58.400972 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25fd1fbe-a44b-47cd-976c-41051459cd22 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:58.463575 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-25fd1fbe-a44b-47cd-976c-41051459cd22 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:58.472171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-25fd1fbe-a44b-47cd-976c-41051459cd22 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.084365 Sep 08 14:01:58.473707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 652/1200] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:58 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:58.846431 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-00ebefe4-d143-49f4-9f1d-0ffd44fd314b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:58.848626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-00ebefe4-d143-49f4-9f1d-0ffd44fd314b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:58.855438 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00ebefe4-d143-49f4-9f1d-0ffd44fd314b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:58.855926 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00ebefe4-d143-49f4-9f1d-0ffd44fd314b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:58.856360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00ebefe4-d143-49f4-9f1d-0ffd44fd314b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:58.857061 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00ebefe4-d143-49f4-9f1d-0ffd44fd314b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:58.857511 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00ebefe4-d143-49f4-9f1d-0ffd44fd314b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:58.857940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00ebefe4-d143-49f4-9f1d-0ffd44fd314b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:58.888063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-00ebefe4-d143-49f4-9f1d-0ffd44fd314b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046304 Sep 08 14:01:58.888666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 549/1201] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:58 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:58.974572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-985cf54d-9cd0-47ec-8784-4e389f3dac5a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:58.977413 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-985cf54d-9cd0-47ec-8784-4e389f3dac5a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:58.984100 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-985cf54d-9cd0-47ec-8784-4e389f3dac5a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:58.984592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-985cf54d-9cd0-47ec-8784-4e389f3dac5a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:58.985032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-985cf54d-9cd0-47ec-8784-4e389f3dac5a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:58.985729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-985cf54d-9cd0-47ec-8784-4e389f3dac5a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:58.986231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-985cf54d-9cd0-47ec-8784-4e389f3dac5a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:58.986798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-985cf54d-9cd0-47ec-8784-4e389f3dac5a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:59.017962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-985cf54d-9cd0-47ec-8784-4e389f3dac5a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047598 Sep 08 14:01:59.018582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 653/1202] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:58 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:01:59.370638 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f5e1180c-77b1-424f-bdde-fb1c99a0f89f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:59.372899 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f5e1180c-77b1-424f-bdde-fb1c99a0f89f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:59.379345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f5e1180c-77b1-424f-bdde-fb1c99a0f89f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:59.379828 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f5e1180c-77b1-424f-bdde-fb1c99a0f89f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:59.380538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f5e1180c-77b1-424f-bdde-fb1c99a0f89f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:59.381226 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f5e1180c-77b1-424f-bdde-fb1c99a0f89f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:59.381679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f5e1180c-77b1-424f-bdde-fb1c99a0f89f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:59.382229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f5e1180c-77b1-424f-bdde-fb1c99a0f89f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:59.412143 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f5e1180c-77b1-424f-bdde-fb1c99a0f89f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045713 Sep 08 14:01:59.412716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 550/1203] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:01:59 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:01:59.495532 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-825c1cbe-693e-45e8-9034-a2dcbb193f67 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:59.496219 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-825c1cbe-693e-45e8-9034-a2dcbb193f67 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:59.503479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-825c1cbe-693e-45e8-9034-a2dcbb193f67 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:59.503968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-825c1cbe-693e-45e8-9034-a2dcbb193f67 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:59.504413 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-825c1cbe-693e-45e8-9034-a2dcbb193f67 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:59.505371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-825c1cbe-693e-45e8-9034-a2dcbb193f67 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:59.505837 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-825c1cbe-693e-45e8-9034-a2dcbb193f67 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:59.506303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-825c1cbe-693e-45e8-9034-a2dcbb193f67 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:59.572801 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-825c1cbe-693e-45e8-9034-a2dcbb193f67 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:01:59.581327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-825c1cbe-693e-45e8-9034-a2dcbb193f67 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.089640 Sep 08 14:01:59.581915 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 654/1204] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:59 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:01:59.690416 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2b765aac-4713-4479-8776-7b1bf029917e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:01:59.691211 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2b765aac-4713-4479-8776-7b1bf029917e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:59.699869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b765aac-4713-4479-8776-7b1bf029917e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:59.700602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b765aac-4713-4479-8776-7b1bf029917e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:59.701238 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b765aac-4713-4479-8776-7b1bf029917e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:59.702318 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b765aac-4713-4479-8776-7b1bf029917e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:59.702989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b765aac-4713-4479-8776-7b1bf029917e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:59.703699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b765aac-4713-4479-8776-7b1bf029917e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:59.744509 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-2b765aac-4713-4479-8776-7b1bf029917e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lazy-loading 'flavor' on Instance uuid d1062abf-7525-48c5-a9cb-91da3855fd1b {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:01:59.901345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e1799e9b-4fcd-4071-9b92-61a816ae1245 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:59.903814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e1799e9b-4fcd-4071-9b92-61a816ae1245 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:59.909822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1799e9b-4fcd-4071-9b92-61a816ae1245 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:59.910357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1799e9b-4fcd-4071-9b92-61a816ae1245 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:59.910802 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1799e9b-4fcd-4071-9b92-61a816ae1245 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:59.911511 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1799e9b-4fcd-4071-9b92-61a816ae1245 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:59.911970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1799e9b-4fcd-4071-9b92-61a816ae1245 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:59.912402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1799e9b-4fcd-4071-9b92-61a816ae1245 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:59.943046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e1799e9b-4fcd-4071-9b92-61a816ae1245 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046371 Sep 08 14:01:59.943609 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 655/1205] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:01:59 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:01:59.973433 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2b765aac-4713-4479-8776-7b1bf029917e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "POST /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b/action" status: 202 len: 0 microversion: 2.1 time: 0.433090 Sep 08 14:01:59.974016 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 551/1206] 10.222.0.25 () {60 vars in 1278 bytes} [Thu Sep 8 14:01:59 2022] POST /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b/action => generated 0 bytes in 435 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 08 14:01:59.982680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8acd4e98-7f48-4183-b81a-7cf3e2c5e2c7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:01:59.983284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8acd4e98-7f48-4183-b81a-7cf3e2c5e2c7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:01:59.989841 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8acd4e98-7f48-4183-b81a-7cf3e2c5e2c7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:59.990388 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8acd4e98-7f48-4183-b81a-7cf3e2c5e2c7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:59.990849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8acd4e98-7f48-4183-b81a-7cf3e2c5e2c7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:01:59.991585 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8acd4e98-7f48-4183-b81a-7cf3e2c5e2c7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:01:59.992039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8acd4e98-7f48-4183-b81a-7cf3e2c5e2c7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:01:59.992464 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8acd4e98-7f48-4183-b81a-7cf3e2c5e2c7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:00.030652 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-77688f96-6e60-44a5-a6b0-83a46a218075 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:00.032990 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-77688f96-6e60-44a5-a6b0-83a46a218075 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:00.039664 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77688f96-6e60-44a5-a6b0-83a46a218075 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:00.040156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77688f96-6e60-44a5-a6b0-83a46a218075 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:00.041907 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77688f96-6e60-44a5-a6b0-83a46a218075 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:00.041907 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77688f96-6e60-44a5-a6b0-83a46a218075 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:00.041907 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77688f96-6e60-44a5-a6b0-83a46a218075 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:00.042240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77688f96-6e60-44a5-a6b0-83a46a218075 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:00.074517 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-77688f96-6e60-44a5-a6b0-83a46a218075 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048164 Sep 08 14:02:00.075316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 552/1207] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:00 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:00.131259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8acd4e98-7f48-4183-b81a-7cf3e2c5e2c7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:00.139950 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8acd4e98-7f48-4183-b81a-7cf3e2c5e2c7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.161401 Sep 08 14:02:00.140519 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 656/1208] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:01:59 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:00.425268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-840cfcb8-ba5b-4a11-82f3-3a36b30d0cda tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:00.427432 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-840cfcb8-ba5b-4a11-82f3-3a36b30d0cda tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:00.434309 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-840cfcb8-ba5b-4a11-82f3-3a36b30d0cda tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:00.434783 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-840cfcb8-ba5b-4a11-82f3-3a36b30d0cda tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:00.435209 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-840cfcb8-ba5b-4a11-82f3-3a36b30d0cda tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:00.435900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-840cfcb8-ba5b-4a11-82f3-3a36b30d0cda tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:00.436353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-840cfcb8-ba5b-4a11-82f3-3a36b30d0cda tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:00.436768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-840cfcb8-ba5b-4a11-82f3-3a36b30d0cda tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:00.467234 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-840cfcb8-ba5b-4a11-82f3-3a36b30d0cda tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045916 Sep 08 14:02:00.467807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 553/1209] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:00 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:00.606775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-db759536-5903-40a3-b648-54fc6ca4dbf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:00.607681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-db759536-5903-40a3-b648-54fc6ca4dbf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:00.614804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db759536-5903-40a3-b648-54fc6ca4dbf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:00.615314 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db759536-5903-40a3-b648-54fc6ca4dbf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:00.615750 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db759536-5903-40a3-b648-54fc6ca4dbf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:00.617079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db759536-5903-40a3-b648-54fc6ca4dbf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:00.617527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db759536-5903-40a3-b648-54fc6ca4dbf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:00.617948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db759536-5903-40a3-b648-54fc6ca4dbf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:00.682886 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-db759536-5903-40a3-b648-54fc6ca4dbf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:00.690516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-db759536-5903-40a3-b648-54fc6ca4dbf6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.087799 Sep 08 14:02:00.691070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 657/1210] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:00 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:00.957062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5e595897-5587-4d25-b95d-7c2821cbf183 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:00.959150 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5e595897-5587-4d25-b95d-7c2821cbf183 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:00.965838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e595897-5587-4d25-b95d-7c2821cbf183 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:00.966353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e595897-5587-4d25-b95d-7c2821cbf183 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:00.966781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e595897-5587-4d25-b95d-7c2821cbf183 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:00.967583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e595897-5587-4d25-b95d-7c2821cbf183 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:00.968008 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e595897-5587-4d25-b95d-7c2821cbf183 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:00.968429 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e595897-5587-4d25-b95d-7c2821cbf183 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:00.993830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:00.998707 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5e595897-5587-4d25-b95d-7c2821cbf183 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046227 Sep 08 14:02:00.999272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 554/1211] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:00 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:01.087416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ba91954c-b872-4fb7-801b-570dc7527094 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:01.089646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ba91954c-b872-4fb7-801b-570dc7527094 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:01.096192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba91954c-b872-4fb7-801b-570dc7527094 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:01.096676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba91954c-b872-4fb7-801b-570dc7527094 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:01.097104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba91954c-b872-4fb7-801b-570dc7527094 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:01.097798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba91954c-b872-4fb7-801b-570dc7527094 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:01.098276 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba91954c-b872-4fb7-801b-570dc7527094 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:01.098698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba91954c-b872-4fb7-801b-570dc7527094 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:01.128092 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ba91954c-b872-4fb7-801b-570dc7527094 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045126 Sep 08 14:02:01.128641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 658/1212] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:01 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:01.164557 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f7b08bb7-d5cf-45aa-93ff-3b4f22419513 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:01.165170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f7b08bb7-d5cf-45aa-93ff-3b4f22419513 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:01.171828 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f7b08bb7-d5cf-45aa-93ff-3b4f22419513 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:01.172317 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f7b08bb7-d5cf-45aa-93ff-3b4f22419513 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:01.172750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f7b08bb7-d5cf-45aa-93ff-3b4f22419513 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:01.173450 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f7b08bb7-d5cf-45aa-93ff-3b4f22419513 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:01.173907 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f7b08bb7-d5cf-45aa-93ff-3b4f22419513 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:01.174368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f7b08bb7-d5cf-45aa-93ff-3b4f22419513 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:01.327245 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f7b08bb7-d5cf-45aa-93ff-3b4f22419513 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:01.335688 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f7b08bb7-d5cf-45aa-93ff-3b4f22419513 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.175413 Sep 08 14:02:01.336242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 555/1213] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:01 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:01.480068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3a304e98-be38-47ed-926d-6cdb1da8ea99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:01.482238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3a304e98-be38-47ed-926d-6cdb1da8ea99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:01.488294 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3a304e98-be38-47ed-926d-6cdb1da8ea99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:01.488785 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3a304e98-be38-47ed-926d-6cdb1da8ea99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:01.489228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3a304e98-be38-47ed-926d-6cdb1da8ea99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:01.489925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3a304e98-be38-47ed-926d-6cdb1da8ea99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:01.490425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3a304e98-be38-47ed-926d-6cdb1da8ea99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:01.490856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3a304e98-be38-47ed-926d-6cdb1da8ea99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:01.504484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:01.521279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3a304e98-be38-47ed-926d-6cdb1da8ea99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045124 Sep 08 14:02:01.521774 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 659/1214] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:01 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:01.713051 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1e8cfb38-c11d-4489-a14c-3c292cde863e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:01.713933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1e8cfb38-c11d-4489-a14c-3c292cde863e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:01.720569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1e8cfb38-c11d-4489-a14c-3c292cde863e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:01.720985 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1e8cfb38-c11d-4489-a14c-3c292cde863e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:01.721411 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1e8cfb38-c11d-4489-a14c-3c292cde863e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:01.722104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1e8cfb38-c11d-4489-a14c-3c292cde863e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:01.722573 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1e8cfb38-c11d-4489-a14c-3c292cde863e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:01.722989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1e8cfb38-c11d-4489-a14c-3c292cde863e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:01.789714 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1e8cfb38-c11d-4489-a14c-3c292cde863e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:01.798852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1e8cfb38-c11d-4489-a14c-3c292cde863e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.089788 Sep 08 14:02:01.799468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 556/1215] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:01 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:02.015946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e984a9e1-846b-4e4f-8208-4c9eea109b79 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:02.018165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e984a9e1-846b-4e4f-8208-4c9eea109b79 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:02.024493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e984a9e1-846b-4e4f-8208-4c9eea109b79 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:02.024979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e984a9e1-846b-4e4f-8208-4c9eea109b79 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:02.025450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e984a9e1-846b-4e4f-8208-4c9eea109b79 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:02.026161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e984a9e1-846b-4e4f-8208-4c9eea109b79 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:02.026673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e984a9e1-846b-4e4f-8208-4c9eea109b79 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:02.027096 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e984a9e1-846b-4e4f-8208-4c9eea109b79 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:02.058225 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e984a9e1-846b-4e4f-8208-4c9eea109b79 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046488 Sep 08 14:02:02.058724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 660/1216] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:02 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:02.144656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-33ae9852-9b3b-4c27-916b-777facb08dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:02.144656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-33ae9852-9b3b-4c27-916b-777facb08dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:02.148735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33ae9852-9b3b-4c27-916b-777facb08dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:02.149216 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33ae9852-9b3b-4c27-916b-777facb08dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:02.149675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33ae9852-9b3b-4c27-916b-777facb08dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:02.150392 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33ae9852-9b3b-4c27-916b-777facb08dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:02.150834 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33ae9852-9b3b-4c27-916b-777facb08dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:02.151249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33ae9852-9b3b-4c27-916b-777facb08dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:02.181600 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-33ae9852-9b3b-4c27-916b-777facb08dbe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045385 Sep 08 14:02:02.182159 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 557/1217] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:02 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:02.359414 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8fded496-e79a-443d-850c-30b9402e9eaa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:02.360031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8fded496-e79a-443d-850c-30b9402e9eaa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:02.366679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fded496-e79a-443d-850c-30b9402e9eaa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:02.367168 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fded496-e79a-443d-850c-30b9402e9eaa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:02.367608 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fded496-e79a-443d-850c-30b9402e9eaa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:02.368304 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fded496-e79a-443d-850c-30b9402e9eaa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:02.368762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fded496-e79a-443d-850c-30b9402e9eaa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:02.369192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fded496-e79a-443d-850c-30b9402e9eaa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:02.502934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8fded496-e79a-443d-850c-30b9402e9eaa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:02.511042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8fded496-e79a-443d-850c-30b9402e9eaa tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.155971 Sep 08 14:02:02.511619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 661/1218] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:02 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:02.534444 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fb39db7a-6841-4635-a1ed-d2857c424b41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:02.536577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fb39db7a-6841-4635-a1ed-d2857c424b41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:02.543137 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb39db7a-6841-4635-a1ed-d2857c424b41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:02.543616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb39db7a-6841-4635-a1ed-d2857c424b41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:02.544044 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb39db7a-6841-4635-a1ed-d2857c424b41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:02.544728 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb39db7a-6841-4635-a1ed-d2857c424b41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:02.545195 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb39db7a-6841-4635-a1ed-d2857c424b41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:02.545596 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fb39db7a-6841-4635-a1ed-d2857c424b41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:02.575177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fb39db7a-6841-4635-a1ed-d2857c424b41 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044756 Sep 08 14:02:02.575738 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 558/1219] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:02 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:02.822053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b2727b5e-0d4a-4692-94ea-4faa9866daa5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:02.822679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b2727b5e-0d4a-4692-94ea-4faa9866daa5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:02.829246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b2727b5e-0d4a-4692-94ea-4faa9866daa5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:02.829727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b2727b5e-0d4a-4692-94ea-4faa9866daa5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:02.830184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b2727b5e-0d4a-4692-94ea-4faa9866daa5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:02.831027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b2727b5e-0d4a-4692-94ea-4faa9866daa5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:02.831478 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b2727b5e-0d4a-4692-94ea-4faa9866daa5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:02.831944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b2727b5e-0d4a-4692-94ea-4faa9866daa5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:02.894635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b2727b5e-0d4a-4692-94ea-4faa9866daa5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:02.902472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b2727b5e-0d4a-4692-94ea-4faa9866daa5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.084399 Sep 08 14:02:02.902938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 662/1220] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:02 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:03.070894 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-49d6ba56-5cd3-4c68-b6f5-561f15f81028 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:03.073060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-49d6ba56-5cd3-4c68-b6f5-561f15f81028 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:03.081400 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-49d6ba56-5cd3-4c68-b6f5-561f15f81028 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:03.081896 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-49d6ba56-5cd3-4c68-b6f5-561f15f81028 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:03.082367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-49d6ba56-5cd3-4c68-b6f5-561f15f81028 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:03.083077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-49d6ba56-5cd3-4c68-b6f5-561f15f81028 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:03.083530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-49d6ba56-5cd3-4c68-b6f5-561f15f81028 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:03.083950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-49d6ba56-5cd3-4c68-b6f5-561f15f81028 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:03.115269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-49d6ba56-5cd3-4c68-b6f5-561f15f81028 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048430 Sep 08 14:02:03.115817 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 559/1221] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:03 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:03.194470 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f040a726-4dc2-4166-bcee-f0426d38c02e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:03.196649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f040a726-4dc2-4166-bcee-f0426d38c02e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:03.203045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f040a726-4dc2-4166-bcee-f0426d38c02e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:03.203538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f040a726-4dc2-4166-bcee-f0426d38c02e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:03.203984 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f040a726-4dc2-4166-bcee-f0426d38c02e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:03.205492 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f040a726-4dc2-4166-bcee-f0426d38c02e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:03.206199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f040a726-4dc2-4166-bcee-f0426d38c02e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:03.206621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f040a726-4dc2-4166-bcee-f0426d38c02e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:03.237978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f040a726-4dc2-4166-bcee-f0426d38c02e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047489 Sep 08 14:02:03.238577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 663/1222] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:03 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:03.535603 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d8be4cc7-10ef-4313-948d-b5338a60e76b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:03.536121 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d8be4cc7-10ef-4313-948d-b5338a60e76b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:03.542600 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d8be4cc7-10ef-4313-948d-b5338a60e76b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:03.543081 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d8be4cc7-10ef-4313-948d-b5338a60e76b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:03.543535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d8be4cc7-10ef-4313-948d-b5338a60e76b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:03.544199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d8be4cc7-10ef-4313-948d-b5338a60e76b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:03.544651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d8be4cc7-10ef-4313-948d-b5338a60e76b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:03.545100 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d8be4cc7-10ef-4313-948d-b5338a60e76b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:03.589122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9dd71652-7cc4-492e-b0b1-677445659c17 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:03.591466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9dd71652-7cc4-492e-b0b1-677445659c17 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:03.598210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9dd71652-7cc4-492e-b0b1-677445659c17 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:03.598712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9dd71652-7cc4-492e-b0b1-677445659c17 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:03.599159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9dd71652-7cc4-492e-b0b1-677445659c17 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:03.599866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9dd71652-7cc4-492e-b0b1-677445659c17 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:03.600328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9dd71652-7cc4-492e-b0b1-677445659c17 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:03.600758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9dd71652-7cc4-492e-b0b1-677445659c17 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:03.633628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9dd71652-7cc4-492e-b0b1-677445659c17 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048548 Sep 08 14:02:03.634486 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 664/1223] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:03 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:03.685970 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d8be4cc7-10ef-4313-948d-b5338a60e76b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:03.698411 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d8be4cc7-10ef-4313-948d-b5338a60e76b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.166491 Sep 08 14:02:03.699205 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 560/1224] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:03 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:03.925886 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2b009bfc-99ad-4625-85e3-16569530551d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:03.926520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2b009bfc-99ad-4625-85e3-16569530551d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:03.933009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b009bfc-99ad-4625-85e3-16569530551d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:03.933498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b009bfc-99ad-4625-85e3-16569530551d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:03.934045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b009bfc-99ad-4625-85e3-16569530551d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:03.934779 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b009bfc-99ad-4625-85e3-16569530551d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:03.935242 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b009bfc-99ad-4625-85e3-16569530551d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:03.935675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b009bfc-99ad-4625-85e3-16569530551d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:04.001960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2b009bfc-99ad-4625-85e3-16569530551d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:04.010962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2b009bfc-99ad-4625-85e3-16569530551d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.089018 Sep 08 14:02:04.011524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 665/1225] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:03 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:04.128157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-09d8f6c1-f4be-49a2-a629-5154e25849dd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:04.130518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-09d8f6c1-f4be-49a2-a629-5154e25849dd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:04.137646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-09d8f6c1-f4be-49a2-a629-5154e25849dd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:04.138126 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-09d8f6c1-f4be-49a2-a629-5154e25849dd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:04.138595 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-09d8f6c1-f4be-49a2-a629-5154e25849dd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:04.139289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-09d8f6c1-f4be-49a2-a629-5154e25849dd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:04.139744 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-09d8f6c1-f4be-49a2-a629-5154e25849dd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:04.140171 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-09d8f6c1-f4be-49a2-a629-5154e25849dd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:04.171410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-09d8f6c1-f4be-49a2-a629-5154e25849dd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047401 Sep 08 14:02:04.171977 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 561/1226] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:04 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:04.250714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ebb56ba0-31c3-41dc-a4c0-0765f3d4702d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:04.252865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ebb56ba0-31c3-41dc-a4c0-0765f3d4702d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:04.259260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ebb56ba0-31c3-41dc-a4c0-0765f3d4702d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:04.259745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ebb56ba0-31c3-41dc-a4c0-0765f3d4702d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:04.260178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ebb56ba0-31c3-41dc-a4c0-0765f3d4702d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:04.260973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ebb56ba0-31c3-41dc-a4c0-0765f3d4702d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:04.261561 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ebb56ba0-31c3-41dc-a4c0-0765f3d4702d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:04.261846 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ebb56ba0-31c3-41dc-a4c0-0765f3d4702d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:04.291111 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ebb56ba0-31c3-41dc-a4c0-0765f3d4702d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044366 Sep 08 14:02:04.291659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 666/1227] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:04 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:04.645725 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cddc953c-7965-4a17-993a-0d4b0d30ca47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:04.647899 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cddc953c-7965-4a17-993a-0d4b0d30ca47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:04.654422 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cddc953c-7965-4a17-993a-0d4b0d30ca47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:04.654906 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cddc953c-7965-4a17-993a-0d4b0d30ca47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:04.655342 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cddc953c-7965-4a17-993a-0d4b0d30ca47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:04.656065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cddc953c-7965-4a17-993a-0d4b0d30ca47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:04.656518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cddc953c-7965-4a17-993a-0d4b0d30ca47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:04.656943 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cddc953c-7965-4a17-993a-0d4b0d30ca47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:04.687944 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cddc953c-7965-4a17-993a-0d4b0d30ca47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046171 Sep 08 14:02:04.688516 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 562/1228] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:04 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:04.729327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-da7117f0-7012-4c17-b86d-015cfac19b52 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:04.729936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-da7117f0-7012-4c17-b86d-015cfac19b52 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:04.736311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-da7117f0-7012-4c17-b86d-015cfac19b52 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:04.736794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-da7117f0-7012-4c17-b86d-015cfac19b52 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:04.737231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-da7117f0-7012-4c17-b86d-015cfac19b52 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:04.737919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-da7117f0-7012-4c17-b86d-015cfac19b52 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:04.738402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-da7117f0-7012-4c17-b86d-015cfac19b52 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:04.738828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-da7117f0-7012-4c17-b86d-015cfac19b52 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:04.871212 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-da7117f0-7012-4c17-b86d-015cfac19b52 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:04.879060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-da7117f0-7012-4c17-b86d-015cfac19b52 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.153776 Sep 08 14:02:04.879891 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 667/1229] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:04 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:05.035952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cecccd28-325b-41cd-a56e-f6aa7291c736 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:05.036471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cecccd28-325b-41cd-a56e-f6aa7291c736 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:05.042971 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cecccd28-325b-41cd-a56e-f6aa7291c736 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:05.043450 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cecccd28-325b-41cd-a56e-f6aa7291c736 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:05.043877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cecccd28-325b-41cd-a56e-f6aa7291c736 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:05.044564 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cecccd28-325b-41cd-a56e-f6aa7291c736 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:05.045000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cecccd28-325b-41cd-a56e-f6aa7291c736 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:05.045414 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cecccd28-325b-41cd-a56e-f6aa7291c736 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:05.110090 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-cecccd28-325b-41cd-a56e-f6aa7291c736 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:05.118417 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cecccd28-325b-41cd-a56e-f6aa7291c736 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.086454 Sep 08 14:02:05.118977 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 563/1230] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:05 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:05.183935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-56787dca-50a8-4afe-8d80-058e42f4bc1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:05.186117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-56787dca-50a8-4afe-8d80-058e42f4bc1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:05.193844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56787dca-50a8-4afe-8d80-058e42f4bc1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:05.193844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56787dca-50a8-4afe-8d80-058e42f4bc1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:05.193844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56787dca-50a8-4afe-8d80-058e42f4bc1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:05.193844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56787dca-50a8-4afe-8d80-058e42f4bc1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:05.194317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56787dca-50a8-4afe-8d80-058e42f4bc1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:05.194664 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56787dca-50a8-4afe-8d80-058e42f4bc1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:05.223754 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-56787dca-50a8-4afe-8d80-058e42f4bc1b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044448 Sep 08 14:02:05.224315 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 668/1231] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:05 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:05.303910 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-927f120a-ad77-4a69-8469-95b691168b81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:05.306682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-927f120a-ad77-4a69-8469-95b691168b81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:05.313010 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-927f120a-ad77-4a69-8469-95b691168b81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:05.313496 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-927f120a-ad77-4a69-8469-95b691168b81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:05.313931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-927f120a-ad77-4a69-8469-95b691168b81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:05.314668 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-927f120a-ad77-4a69-8469-95b691168b81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:05.315123 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-927f120a-ad77-4a69-8469-95b691168b81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:05.315577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-927f120a-ad77-4a69-8469-95b691168b81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:05.346392 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-927f120a-ad77-4a69-8469-95b691168b81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046478 Sep 08 14:02:05.347029 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 564/1232] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:05 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:05.700881 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-01694320-bffe-43cb-8b47-1760d6728a9c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:05.703019 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-01694320-bffe-43cb-8b47-1760d6728a9c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:05.709169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01694320-bffe-43cb-8b47-1760d6728a9c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:05.709643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01694320-bffe-43cb-8b47-1760d6728a9c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:05.710071 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01694320-bffe-43cb-8b47-1760d6728a9c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:05.710781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01694320-bffe-43cb-8b47-1760d6728a9c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:05.711222 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01694320-bffe-43cb-8b47-1760d6728a9c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:05.711644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01694320-bffe-43cb-8b47-1760d6728a9c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:05.744619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-01694320-bffe-43cb-8b47-1760d6728a9c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048149 Sep 08 14:02:05.745310 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 669/1233] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:05 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:05.904351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cd24dca9-3522-48e3-a8ec-bf1e357b49dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:05.904875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cd24dca9-3522-48e3-a8ec-bf1e357b49dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:05.911391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd24dca9-3522-48e3-a8ec-bf1e357b49dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:05.911869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd24dca9-3522-48e3-a8ec-bf1e357b49dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:05.912299 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd24dca9-3522-48e3-a8ec-bf1e357b49dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:05.912991 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd24dca9-3522-48e3-a8ec-bf1e357b49dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:05.913440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd24dca9-3522-48e3-a8ec-bf1e357b49dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:05.913860 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd24dca9-3522-48e3-a8ec-bf1e357b49dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:06.047406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-cd24dca9-3522-48e3-a8ec-bf1e357b49dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:06.056112 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cd24dca9-3522-48e3-a8ec-bf1e357b49dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.155710 Sep 08 14:02:06.056928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 565/1234] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:05 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:06.141275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bc5e29e4-7770-4076-9003-3d19f8bdcab8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:06.141878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bc5e29e4-7770-4076-9003-3d19f8bdcab8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:06.147768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bc5e29e4-7770-4076-9003-3d19f8bdcab8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:06.148245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bc5e29e4-7770-4076-9003-3d19f8bdcab8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:06.148677 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bc5e29e4-7770-4076-9003-3d19f8bdcab8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:06.149356 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bc5e29e4-7770-4076-9003-3d19f8bdcab8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:06.149806 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bc5e29e4-7770-4076-9003-3d19f8bdcab8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:06.150256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bc5e29e4-7770-4076-9003-3d19f8bdcab8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:06.210931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bc5e29e4-7770-4076-9003-3d19f8bdcab8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:06.218922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bc5e29e4-7770-4076-9003-3d19f8bdcab8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.081537 Sep 08 14:02:06.219487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 670/1235] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:06 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:06.237239 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0061efaa-3e8d-407b-80d3-bd526647fb3b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:06.239535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0061efaa-3e8d-407b-80d3-bd526647fb3b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:06.241509 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:06.246861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0061efaa-3e8d-407b-80d3-bd526647fb3b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:06.251930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0061efaa-3e8d-407b-80d3-bd526647fb3b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:06.251930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0061efaa-3e8d-407b-80d3-bd526647fb3b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:06.251930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0061efaa-3e8d-407b-80d3-bd526647fb3b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:06.251930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0061efaa-3e8d-407b-80d3-bd526647fb3b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:06.252353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0061efaa-3e8d-407b-80d3-bd526647fb3b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:06.279597 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0061efaa-3e8d-407b-80d3-bd526647fb3b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046873 Sep 08 14:02:06.280154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 566/1236] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:06 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:06.359491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c86aa74f-bb22-4a20-ad19-89f8f63562a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:06.361637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c86aa74f-bb22-4a20-ad19-89f8f63562a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:06.367621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c86aa74f-bb22-4a20-ad19-89f8f63562a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:06.368658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c86aa74f-bb22-4a20-ad19-89f8f63562a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:06.369102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c86aa74f-bb22-4a20-ad19-89f8f63562a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:06.369796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c86aa74f-bb22-4a20-ad19-89f8f63562a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:06.370290 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c86aa74f-bb22-4a20-ad19-89f8f63562a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:06.370718 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c86aa74f-bb22-4a20-ad19-89f8f63562a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:06.400509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c86aa74f-bb22-4a20-ad19-89f8f63562a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045229 Sep 08 14:02:06.401185 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 671/1237] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:06 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:06.757881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-dfc22a6a-837c-4e2b-837b-c0fdc0941a47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:06.760076 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-dfc22a6a-837c-4e2b-837b-c0fdc0941a47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:06.766087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dfc22a6a-837c-4e2b-837b-c0fdc0941a47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:06.766593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dfc22a6a-837c-4e2b-837b-c0fdc0941a47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:06.767043 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dfc22a6a-837c-4e2b-837b-c0fdc0941a47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:06.767701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dfc22a6a-837c-4e2b-837b-c0fdc0941a47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:06.768143 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dfc22a6a-837c-4e2b-837b-c0fdc0941a47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:06.768555 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dfc22a6a-837c-4e2b-837b-c0fdc0941a47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:06.798200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-dfc22a6a-837c-4e2b-837b-c0fdc0941a47 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044761 Sep 08 14:02:06.798751 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 567/1238] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:06 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:07.082243 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8ddb08bf-f8e5-40a4-b5c9-ed1ffef27496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:07.083531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8ddb08bf-f8e5-40a4-b5c9-ed1ffef27496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:07.095761 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8ddb08bf-f8e5-40a4-b5c9-ed1ffef27496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:07.096316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8ddb08bf-f8e5-40a4-b5c9-ed1ffef27496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:07.096875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8ddb08bf-f8e5-40a4-b5c9-ed1ffef27496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:07.098449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8ddb08bf-f8e5-40a4-b5c9-ed1ffef27496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:07.098915 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8ddb08bf-f8e5-40a4-b5c9-ed1ffef27496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:07.099345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8ddb08bf-f8e5-40a4-b5c9-ed1ffef27496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:07.246573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8ddb08bf-f8e5-40a4-b5c9-ed1ffef27496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:07.248348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-89118f51-0345-456d-b8d1-7883e525760d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:07.248960 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-89118f51-0345-456d-b8d1-7883e525760d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:07.255162 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8ddb08bf-f8e5-40a4-b5c9-ed1ffef27496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.177727 Sep 08 14:02:07.255845 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 672/1239] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:07 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:07.257111 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89118f51-0345-456d-b8d1-7883e525760d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:07.257611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89118f51-0345-456d-b8d1-7883e525760d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:07.258046 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89118f51-0345-456d-b8d1-7883e525760d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:07.259322 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89118f51-0345-456d-b8d1-7883e525760d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:07.259765 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89118f51-0345-456d-b8d1-7883e525760d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:07.260190 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89118f51-0345-456d-b8d1-7883e525760d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:07.294874 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d872aed8-ef7d-4d6d-b2f7-ac09084a6571 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:07.297042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d872aed8-ef7d-4d6d-b2f7-ac09084a6571 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:07.303746 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d872aed8-ef7d-4d6d-b2f7-ac09084a6571 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:07.304448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d872aed8-ef7d-4d6d-b2f7-ac09084a6571 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:07.304967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d872aed8-ef7d-4d6d-b2f7-ac09084a6571 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:07.305662 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d872aed8-ef7d-4d6d-b2f7-ac09084a6571 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:07.306121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d872aed8-ef7d-4d6d-b2f7-ac09084a6571 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:07.306576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d872aed8-ef7d-4d6d-b2f7-ac09084a6571 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:07.330405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-89118f51-0345-456d-b8d1-7883e525760d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:07.339045 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-89118f51-0345-456d-b8d1-7883e525760d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.094662 Sep 08 14:02:07.342556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d872aed8-ef7d-4d6d-b2f7-ac09084a6571 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048992 Sep 08 14:02:07.342749 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 568/1240] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:07 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:07.342855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 673/1241] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:07 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:07.413245 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e006124f-1561-48ab-a3f2-5f8ede430f87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:07.415423 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e006124f-1561-48ab-a3f2-5f8ede430f87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:07.421782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e006124f-1561-48ab-a3f2-5f8ede430f87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:07.422305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e006124f-1561-48ab-a3f2-5f8ede430f87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:07.422969 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e006124f-1561-48ab-a3f2-5f8ede430f87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:07.423563 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e006124f-1561-48ab-a3f2-5f8ede430f87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:07.424018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e006124f-1561-48ab-a3f2-5f8ede430f87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:07.424437 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e006124f-1561-48ab-a3f2-5f8ede430f87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:07.455891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e006124f-1561-48ab-a3f2-5f8ede430f87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046574 Sep 08 14:02:07.456466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 569/1242] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:07 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:07.811457 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0871772e-0a71-4bbf-a690-e8fcfc845721 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:07.813616 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0871772e-0a71-4bbf-a690-e8fcfc845721 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:07.819575 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0871772e-0a71-4bbf-a690-e8fcfc845721 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:07.820059 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0871772e-0a71-4bbf-a690-e8fcfc845721 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:07.820496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0871772e-0a71-4bbf-a690-e8fcfc845721 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:07.821180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0871772e-0a71-4bbf-a690-e8fcfc845721 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:07.822232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0871772e-0a71-4bbf-a690-e8fcfc845721 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:07.822660 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0871772e-0a71-4bbf-a690-e8fcfc845721 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:07.851403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0871772e-0a71-4bbf-a690-e8fcfc845721 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044042 Sep 08 14:02:07.851947 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 674/1243] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:07 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:08.281175 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-36becf96-e89c-405c-925e-e46878eda7c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:08.281786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-36becf96-e89c-405c-925e-e46878eda7c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:08.288238 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-36becf96-e89c-405c-925e-e46878eda7c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:08.288717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-36becf96-e89c-405c-925e-e46878eda7c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:08.289160 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-36becf96-e89c-405c-925e-e46878eda7c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:08.289845 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-36becf96-e89c-405c-925e-e46878eda7c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:08.290935 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-36becf96-e89c-405c-925e-e46878eda7c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:08.291366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-36becf96-e89c-405c-925e-e46878eda7c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:08.353232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f461411f-f55d-45c9-a7a8-a922258c54fa tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:08.356208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f461411f-f55d-45c9-a7a8-a922258c54fa tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:08.363298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f461411f-f55d-45c9-a7a8-a922258c54fa tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:08.363869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f461411f-f55d-45c9-a7a8-a922258c54fa tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:08.364309 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f461411f-f55d-45c9-a7a8-a922258c54fa tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:08.365008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f461411f-f55d-45c9-a7a8-a922258c54fa tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:08.365458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f461411f-f55d-45c9-a7a8-a922258c54fa tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:08.365884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f461411f-f55d-45c9-a7a8-a922258c54fa tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:08.404762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f461411f-f55d-45c9-a7a8-a922258c54fa tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056276 Sep 08 14:02:08.405341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 675/1244] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:08 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:08.409910 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1fcb54ac-10e9-42ad-b033-b7d003b54052 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:08.410532 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1fcb54ac-10e9-42ad-b033-b7d003b54052 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:08.416956 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1fcb54ac-10e9-42ad-b033-b7d003b54052 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:08.417503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1fcb54ac-10e9-42ad-b033-b7d003b54052 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:08.417939 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1fcb54ac-10e9-42ad-b033-b7d003b54052 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:08.418664 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1fcb54ac-10e9-42ad-b033-b7d003b54052 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:08.419116 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1fcb54ac-10e9-42ad-b033-b7d003b54052 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:08.419536 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1fcb54ac-10e9-42ad-b033-b7d003b54052 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:08.423608 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:08.427699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-36becf96-e89c-405c-925e-e46878eda7c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:08.435705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-36becf96-e89c-405c-925e-e46878eda7c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.158421 Sep 08 14:02:08.437704 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 570/1245] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:08 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:08.469276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-969e176f-0faf-468c-bf9e-8207ee7448a1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:08.471473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-969e176f-0faf-468c-bf9e-8207ee7448a1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:08.478037 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-969e176f-0faf-468c-bf9e-8207ee7448a1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:08.478492 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-969e176f-0faf-468c-bf9e-8207ee7448a1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:08.478927 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-969e176f-0faf-468c-bf9e-8207ee7448a1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:08.479619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-969e176f-0faf-468c-bf9e-8207ee7448a1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:08.480059 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-969e176f-0faf-468c-bf9e-8207ee7448a1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:08.480478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-969e176f-0faf-468c-bf9e-8207ee7448a1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:08.481953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1fcb54ac-10e9-42ad-b033-b7d003b54052 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:08.489814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1fcb54ac-10e9-42ad-b033-b7d003b54052 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.083506 Sep 08 14:02:08.490415 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 676/1246] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:08 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:08.497291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:08.510169 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-969e176f-0faf-468c-bf9e-8207ee7448a1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045137 Sep 08 14:02:08.511487 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 571/1247] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:08 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:08.864175 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-df22fcf1-a672-4124-aa8e-2f8f5a851c33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:08.866347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-df22fcf1-a672-4124-aa8e-2f8f5a851c33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:08.872666 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df22fcf1-a672-4124-aa8e-2f8f5a851c33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:08.873158 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df22fcf1-a672-4124-aa8e-2f8f5a851c33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:08.873599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df22fcf1-a672-4124-aa8e-2f8f5a851c33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:08.874349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df22fcf1-a672-4124-aa8e-2f8f5a851c33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:08.874810 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df22fcf1-a672-4124-aa8e-2f8f5a851c33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:08.875240 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df22fcf1-a672-4124-aa8e-2f8f5a851c33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:08.905944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-df22fcf1-a672-4124-aa8e-2f8f5a851c33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045744 Sep 08 14:02:08.906455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 677/1248] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:08 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:09.417837 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5b001016-9460-4a5f-9a60-73c1b53b61c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:09.420041 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5b001016-9460-4a5f-9a60-73c1b53b61c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:09.426513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5b001016-9460-4a5f-9a60-73c1b53b61c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:09.426995 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5b001016-9460-4a5f-9a60-73c1b53b61c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:09.427425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5b001016-9460-4a5f-9a60-73c1b53b61c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:09.428105 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5b001016-9460-4a5f-9a60-73c1b53b61c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:09.428552 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5b001016-9460-4a5f-9a60-73c1b53b61c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:09.428972 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5b001016-9460-4a5f-9a60-73c1b53b61c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:09.458465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5b001016-9460-4a5f-9a60-73c1b53b61c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045313 Sep 08 14:02:09.458938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5f1db7b9-ec51-43ab-a644-2695a0f974cc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:09.459393 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 572/1249] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:09 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:09.459575 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5f1db7b9-ec51-43ab-a644-2695a0f974cc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:09.465979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f1db7b9-ec51-43ab-a644-2695a0f974cc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:09.466504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f1db7b9-ec51-43ab-a644-2695a0f974cc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:09.466937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f1db7b9-ec51-43ab-a644-2695a0f974cc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:09.467627 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f1db7b9-ec51-43ab-a644-2695a0f974cc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:09.468072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f1db7b9-ec51-43ab-a644-2695a0f974cc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:09.468490 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f1db7b9-ec51-43ab-a644-2695a0f974cc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:09.512824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-94cbca8e-66ee-452f-a505-7b2478000378 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:09.513424 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-94cbca8e-66ee-452f-a505-7b2478000378 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:09.519340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94cbca8e-66ee-452f-a505-7b2478000378 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:09.519813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94cbca8e-66ee-452f-a505-7b2478000378 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:09.520240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94cbca8e-66ee-452f-a505-7b2478000378 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:09.520919 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94cbca8e-66ee-452f-a505-7b2478000378 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:09.521359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94cbca8e-66ee-452f-a505-7b2478000378 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:09.521770 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94cbca8e-66ee-452f-a505-7b2478000378 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:09.582816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-94cbca8e-66ee-452f-a505-7b2478000378 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:09.591228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-94cbca8e-66ee-452f-a505-7b2478000378 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.082217 Sep 08 14:02:09.591798 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 573/1250] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:09 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:09.601735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5f1db7b9-ec51-43ab-a644-2695a0f974cc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:09.604975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-21e97af9-3589-4e6f-beaf-743e5b0f7d6f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:09.607198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-21e97af9-3589-4e6f-beaf-743e5b0f7d6f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:09.610316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5f1db7b9-ec51-43ab-a644-2695a0f974cc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.155218 Sep 08 14:02:09.610893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 678/1251] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:09 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:09.613077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21e97af9-3589-4e6f-beaf-743e5b0f7d6f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:09.613560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21e97af9-3589-4e6f-beaf-743e5b0f7d6f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:09.613993 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21e97af9-3589-4e6f-beaf-743e5b0f7d6f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:09.614737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21e97af9-3589-4e6f-beaf-743e5b0f7d6f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:09.615184 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21e97af9-3589-4e6f-beaf-743e5b0f7d6f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:09.615604 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21e97af9-3589-4e6f-beaf-743e5b0f7d6f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:09.644941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-21e97af9-3589-4e6f-beaf-743e5b0f7d6f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044438 Sep 08 14:02:09.645499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 574/1252] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:09 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:09.919235 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f56ca3d8-7223-4a4a-a8f6-048ea84f893a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:09.921996 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f56ca3d8-7223-4a4a-a8f6-048ea84f893a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:09.928012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f56ca3d8-7223-4a4a-a8f6-048ea84f893a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:09.928431 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f56ca3d8-7223-4a4a-a8f6-048ea84f893a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:09.928869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f56ca3d8-7223-4a4a-a8f6-048ea84f893a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:09.929564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f56ca3d8-7223-4a4a-a8f6-048ea84f893a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:09.930016 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f56ca3d8-7223-4a4a-a8f6-048ea84f893a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:09.930468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f56ca3d8-7223-4a4a-a8f6-048ea84f893a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:09.959407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f56ca3d8-7223-4a4a-a8f6-048ea84f893a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044859 Sep 08 14:02:09.959958 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 679/1253] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:09 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:10.471020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8eeb3744-5014-4c24-9abc-8547f58d2ff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:10.473166 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8eeb3744-5014-4c24-9abc-8547f58d2ff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:10.479583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8eeb3744-5014-4c24-9abc-8547f58d2ff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:10.480062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8eeb3744-5014-4c24-9abc-8547f58d2ff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:10.480480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8eeb3744-5014-4c24-9abc-8547f58d2ff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:10.481176 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8eeb3744-5014-4c24-9abc-8547f58d2ff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:10.481618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8eeb3744-5014-4c24-9abc-8547f58d2ff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:10.482039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8eeb3744-5014-4c24-9abc-8547f58d2ff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:10.511846 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8eeb3744-5014-4c24-9abc-8547f58d2ff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044756 Sep 08 14:02:10.512414 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 575/1254] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:10 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:10.614016 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-090150ec-0a06-4fdc-8210-1e64b2e2f673 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:10.614611 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-090150ec-0a06-4fdc-8210-1e64b2e2f673 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:10.622389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-090150ec-0a06-4fdc-8210-1e64b2e2f673 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:10.622878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-090150ec-0a06-4fdc-8210-1e64b2e2f673 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:10.623310 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-090150ec-0a06-4fdc-8210-1e64b2e2f673 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:10.624004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-090150ec-0a06-4fdc-8210-1e64b2e2f673 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:10.624453 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-090150ec-0a06-4fdc-8210-1e64b2e2f673 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:10.624873 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-090150ec-0a06-4fdc-8210-1e64b2e2f673 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:10.638310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6374e2bb-0f4d-4a54-9fa0-c5c197cdffce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:10.638310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6374e2bb-0f4d-4a54-9fa0-c5c197cdffce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:10.642940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6374e2bb-0f4d-4a54-9fa0-c5c197cdffce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:10.643433 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6374e2bb-0f4d-4a54-9fa0-c5c197cdffce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:10.643871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6374e2bb-0f4d-4a54-9fa0-c5c197cdffce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:10.644647 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6374e2bb-0f4d-4a54-9fa0-c5c197cdffce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:10.645090 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6374e2bb-0f4d-4a54-9fa0-c5c197cdffce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:10.645515 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6374e2bb-0f4d-4a54-9fa0-c5c197cdffce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:10.699092 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-090150ec-0a06-4fdc-8210-1e64b2e2f673 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:10.706764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-090150ec-0a06-4fdc-8210-1e64b2e2f673 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.096750 Sep 08 14:02:10.707496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 680/1255] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:10 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:10.712154 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-75cb5479-ab13-451e-ba8a-8d414ba02198 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:10.714650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-75cb5479-ab13-451e-ba8a-8d414ba02198 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:10.721206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75cb5479-ab13-451e-ba8a-8d414ba02198 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:10.721769 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75cb5479-ab13-451e-ba8a-8d414ba02198 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:10.722233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75cb5479-ab13-451e-ba8a-8d414ba02198 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:10.722945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75cb5479-ab13-451e-ba8a-8d414ba02198 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:10.723403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75cb5479-ab13-451e-ba8a-8d414ba02198 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:10.723822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75cb5479-ab13-451e-ba8a-8d414ba02198 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:10.755901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-75cb5479-ab13-451e-ba8a-8d414ba02198 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047355 Sep 08 14:02:10.756474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 681/1256] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:10 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:10.862371 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6374e2bb-0f4d-4a54-9fa0-c5c197cdffce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:10.871076 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6374e2bb-0f4d-4a54-9fa0-c5c197cdffce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.240378 Sep 08 14:02:10.871760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 576/1257] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:10 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:10.973037 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ef0e9db1-9ef8-4e17-9c79-23bfbfa5bcf2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:10.975254 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ef0e9db1-9ef8-4e17-9c79-23bfbfa5bcf2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:10.982205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef0e9db1-9ef8-4e17-9c79-23bfbfa5bcf2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:10.982695 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef0e9db1-9ef8-4e17-9c79-23bfbfa5bcf2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:10.983135 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef0e9db1-9ef8-4e17-9c79-23bfbfa5bcf2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:10.983852 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef0e9db1-9ef8-4e17-9c79-23bfbfa5bcf2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:10.984338 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef0e9db1-9ef8-4e17-9c79-23bfbfa5bcf2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:10.984729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef0e9db1-9ef8-4e17-9c79-23bfbfa5bcf2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:11.018302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ef0e9db1-9ef8-4e17-9c79-23bfbfa5bcf2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049230 Sep 08 14:02:11.024914 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 682/1258] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:10 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:11.525829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-806220c2-fc00-4cf9-960f-ad6a1bcc27dc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:11.527998 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-806220c2-fc00-4cf9-960f-ad6a1bcc27dc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:11.534700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-806220c2-fc00-4cf9-960f-ad6a1bcc27dc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:11.535183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-806220c2-fc00-4cf9-960f-ad6a1bcc27dc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:11.535615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-806220c2-fc00-4cf9-960f-ad6a1bcc27dc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:11.536438 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-806220c2-fc00-4cf9-960f-ad6a1bcc27dc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:11.536888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-806220c2-fc00-4cf9-960f-ad6a1bcc27dc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:11.537309 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-806220c2-fc00-4cf9-960f-ad6a1bcc27dc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:11.569588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-806220c2-fc00-4cf9-960f-ad6a1bcc27dc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048392 Sep 08 14:02:11.570116 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 577/1259] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:11 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:11.731079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-93826d17-fe61-441c-acfd-4f4a0eb32879 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:11.731721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-93826d17-fe61-441c-acfd-4f4a0eb32879 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:11.738463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93826d17-fe61-441c-acfd-4f4a0eb32879 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:11.738957 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93826d17-fe61-441c-acfd-4f4a0eb32879 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:11.739400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93826d17-fe61-441c-acfd-4f4a0eb32879 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:11.740109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93826d17-fe61-441c-acfd-4f4a0eb32879 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:11.740564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93826d17-fe61-441c-acfd-4f4a0eb32879 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:11.740993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93826d17-fe61-441c-acfd-4f4a0eb32879 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:11.771448 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ae181283-6504-48cd-ba8d-a0ba40a0d5e8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:11.774485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ae181283-6504-48cd-ba8d-a0ba40a0d5e8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:11.781159 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ae181283-6504-48cd-ba8d-a0ba40a0d5e8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:11.781549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ae181283-6504-48cd-ba8d-a0ba40a0d5e8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:11.781983 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ae181283-6504-48cd-ba8d-a0ba40a0d5e8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:11.782725 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ae181283-6504-48cd-ba8d-a0ba40a0d5e8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:11.783177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ae181283-6504-48cd-ba8d-a0ba40a0d5e8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:11.783828 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ae181283-6504-48cd-ba8d-a0ba40a0d5e8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:11.814514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-93826d17-fe61-441c-acfd-4f4a0eb32879 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:11.816642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ae181283-6504-48cd-ba8d-a0ba40a0d5e8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049605 Sep 08 14:02:11.817228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 578/1260] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:11 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:11.822957 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-93826d17-fe61-441c-acfd-4f4a0eb32879 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.095925 Sep 08 14:02:11.823526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 683/1261] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:11 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:11.895148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-21bb8ee6-3f8f-4666-9fdb-ed0e49e7c55c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:11.895759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-21bb8ee6-3f8f-4666-9fdb-ed0e49e7c55c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:11.902663 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21bb8ee6-3f8f-4666-9fdb-ed0e49e7c55c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:11.903064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21bb8ee6-3f8f-4666-9fdb-ed0e49e7c55c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:11.903494 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21bb8ee6-3f8f-4666-9fdb-ed0e49e7c55c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:11.904193 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21bb8ee6-3f8f-4666-9fdb-ed0e49e7c55c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:11.904826 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21bb8ee6-3f8f-4666-9fdb-ed0e49e7c55c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:11.905058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21bb8ee6-3f8f-4666-9fdb-ed0e49e7c55c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:12.036456 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f02d3de5-7ae0-41a2-8093-033e7866ec8c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:12.038565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f02d3de5-7ae0-41a2-8093-033e7866ec8c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:12.044978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f02d3de5-7ae0-41a2-8093-033e7866ec8c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:12.045480 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f02d3de5-7ae0-41a2-8093-033e7866ec8c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:12.045922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f02d3de5-7ae0-41a2-8093-033e7866ec8c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:12.046954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f02d3de5-7ae0-41a2-8093-033e7866ec8c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:12.047133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f02d3de5-7ae0-41a2-8093-033e7866ec8c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:12.047572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f02d3de5-7ae0-41a2-8093-033e7866ec8c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:12.053986 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-21bb8ee6-3f8f-4666-9fdb-ed0e49e7c55c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:12.063390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-21bb8ee6-3f8f-4666-9fdb-ed0e49e7c55c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.171715 Sep 08 14:02:12.064093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 579/1262] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:11 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:12.095399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f02d3de5-7ae0-41a2-8093-033e7866ec8c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.063356 Sep 08 14:02:12.095977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 684/1263] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:12 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:02:12.582419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0d93c2f4-97a9-40af-8bc3-c8ff1c77432a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:12.584731 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0d93c2f4-97a9-40af-8bc3-c8ff1c77432a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:12.592243 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0d93c2f4-97a9-40af-8bc3-c8ff1c77432a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:12.592727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0d93c2f4-97a9-40af-8bc3-c8ff1c77432a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:12.593191 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0d93c2f4-97a9-40af-8bc3-c8ff1c77432a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:12.593882 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0d93c2f4-97a9-40af-8bc3-c8ff1c77432a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:12.594371 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0d93c2f4-97a9-40af-8bc3-c8ff1c77432a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:12.594787 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0d93c2f4-97a9-40af-8bc3-c8ff1c77432a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:12.625301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0d93c2f4-97a9-40af-8bc3-c8ff1c77432a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046824 Sep 08 14:02:12.625858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 580/1264] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:12 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:12.830652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-04b1eb1b-40a1-4b00-b7f8-84454282158e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:12.832958 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-04b1eb1b-40a1-4b00-b7f8-84454282158e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:12.839839 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04b1eb1b-40a1-4b00-b7f8-84454282158e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:12.840324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04b1eb1b-40a1-4b00-b7f8-84454282158e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:12.840759 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04b1eb1b-40a1-4b00-b7f8-84454282158e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:12.841554 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04b1eb1b-40a1-4b00-b7f8-84454282158e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:12.842006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04b1eb1b-40a1-4b00-b7f8-84454282158e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:12.842459 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04b1eb1b-40a1-4b00-b7f8-84454282158e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:12.846420 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1f9bcd4c-9621-4e0d-a471-8e42f98cdb2f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:12.847031 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1f9bcd4c-9621-4e0d-a471-8e42f98cdb2f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:12.853293 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f9bcd4c-9621-4e0d-a471-8e42f98cdb2f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:12.853780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f9bcd4c-9621-4e0d-a471-8e42f98cdb2f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:12.854254 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f9bcd4c-9621-4e0d-a471-8e42f98cdb2f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:12.854975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f9bcd4c-9621-4e0d-a471-8e42f98cdb2f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:12.855611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f9bcd4c-9621-4e0d-a471-8e42f98cdb2f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:12.856042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f9bcd4c-9621-4e0d-a471-8e42f98cdb2f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:12.877735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-04b1eb1b-40a1-4b00-b7f8-84454282158e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051260 Sep 08 14:02:12.878019 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 685/1265] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:12 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:12.921850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1f9bcd4c-9621-4e0d-a471-8e42f98cdb2f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:12.929635 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1f9bcd4c-9621-4e0d-a471-8e42f98cdb2f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.087142 Sep 08 14:02:12.930762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 581/1266] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:12 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:13.087217 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-da05e255-c531-4545-bd8c-9b4a21690411 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:13.087924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-da05e255-c531-4545-bd8c-9b4a21690411 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:13.095373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-da05e255-c531-4545-bd8c-9b4a21690411 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:13.095871 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-da05e255-c531-4545-bd8c-9b4a21690411 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:13.096311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-da05e255-c531-4545-bd8c-9b4a21690411 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:13.097036 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-da05e255-c531-4545-bd8c-9b4a21690411 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:13.097496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-da05e255-c531-4545-bd8c-9b4a21690411 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:13.097927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-da05e255-c531-4545-bd8c-9b4a21690411 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:13.108723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f3edeff2-c401-41af-9f37-5dc2f10f004e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:13.111273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f3edeff2-c401-41af-9f37-5dc2f10f004e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:13.118589 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f3edeff2-c401-41af-9f37-5dc2f10f004e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:13.119075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f3edeff2-c401-41af-9f37-5dc2f10f004e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:13.119511 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f3edeff2-c401-41af-9f37-5dc2f10f004e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:13.120213 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f3edeff2-c401-41af-9f37-5dc2f10f004e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:13.120661 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f3edeff2-c401-41af-9f37-5dc2f10f004e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:13.121079 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f3edeff2-c401-41af-9f37-5dc2f10f004e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:13.152636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f3edeff2-c401-41af-9f37-5dc2f10f004e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048124 Sep 08 14:02:13.153218 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 582/1267] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:13 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:13.231959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-da05e255-c531-4545-bd8c-9b4a21690411 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:13.239816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-da05e255-c531-4545-bd8c-9b4a21690411 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.156535 Sep 08 14:02:13.240413 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 686/1268] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:13 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:13.638583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ce80cc45-92db-4f43-bc26-a59146d844be tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:13.640697 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ce80cc45-92db-4f43-bc26-a59146d844be tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:13.647218 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce80cc45-92db-4f43-bc26-a59146d844be tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:13.647699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce80cc45-92db-4f43-bc26-a59146d844be tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:13.648132 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce80cc45-92db-4f43-bc26-a59146d844be tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:13.649338 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce80cc45-92db-4f43-bc26-a59146d844be tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:13.649786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce80cc45-92db-4f43-bc26-a59146d844be tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:13.650233 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce80cc45-92db-4f43-bc26-a59146d844be tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:13.681259 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ce80cc45-92db-4f43-bc26-a59146d844be tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046730 Sep 08 14:02:13.681825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 583/1269] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:13 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:13.890226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1707fa13-f3a3-4f6d-a562-cf7edb435dc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:13.892364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1707fa13-f3a3-4f6d-a562-cf7edb435dc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:13.899475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1707fa13-f3a3-4f6d-a562-cf7edb435dc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:13.899958 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1707fa13-f3a3-4f6d-a562-cf7edb435dc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:13.900388 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1707fa13-f3a3-4f6d-a562-cf7edb435dc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:13.901080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1707fa13-f3a3-4f6d-a562-cf7edb435dc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:13.901530 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1707fa13-f3a3-4f6d-a562-cf7edb435dc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:13.901951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1707fa13-f3a3-4f6d-a562-cf7edb435dc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:13.934226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1707fa13-f3a3-4f6d-a562-cf7edb435dc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047969 Sep 08 14:02:13.934853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 687/1270] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:13 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:13.954437 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0ff34178-9996-4f83-8af3-9865b9c273a3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:13.955043 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0ff34178-9996-4f83-8af3-9865b9c273a3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:13.961974 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ff34178-9996-4f83-8af3-9865b9c273a3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:13.962596 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ff34178-9996-4f83-8af3-9865b9c273a3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:13.963105 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ff34178-9996-4f83-8af3-9865b9c273a3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:13.964236 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ff34178-9996-4f83-8af3-9865b9c273a3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:13.964697 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ff34178-9996-4f83-8af3-9865b9c273a3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:13.965121 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ff34178-9996-4f83-8af3-9865b9c273a3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:14.037090 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0ff34178-9996-4f83-8af3-9865b9c273a3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:14.045259 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0ff34178-9996-4f83-8af3-9865b9c273a3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.096070 Sep 08 14:02:14.045854 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 584/1271] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:13 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:14.165088 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e00e06c8-204f-4010-938b-20fba4fd26a2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:14.167207 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e00e06c8-204f-4010-938b-20fba4fd26a2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:14.173267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e00e06c8-204f-4010-938b-20fba4fd26a2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:14.173749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e00e06c8-204f-4010-938b-20fba4fd26a2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:14.174207 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e00e06c8-204f-4010-938b-20fba4fd26a2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:14.174900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e00e06c8-204f-4010-938b-20fba4fd26a2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:14.175343 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e00e06c8-204f-4010-938b-20fba4fd26a2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:14.175761 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e00e06c8-204f-4010-938b-20fba4fd26a2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:14.206350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e00e06c8-204f-4010-938b-20fba4fd26a2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045452 Sep 08 14:02:14.206901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 688/1272] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:14 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:14.264555 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6c74e507-fa80-4458-a672-5bf3f30e242a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:14.265175 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6c74e507-fa80-4458-a672-5bf3f30e242a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:14.271158 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c74e507-fa80-4458-a672-5bf3f30e242a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:14.271629 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c74e507-fa80-4458-a672-5bf3f30e242a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:14.272057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c74e507-fa80-4458-a672-5bf3f30e242a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:14.272738 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c74e507-fa80-4458-a672-5bf3f30e242a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:14.273177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c74e507-fa80-4458-a672-5bf3f30e242a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:14.274191 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c74e507-fa80-4458-a672-5bf3f30e242a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:14.415395 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6c74e507-fa80-4458-a672-5bf3f30e242a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:14.423821 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6c74e507-fa80-4458-a672-5bf3f30e242a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.163122 Sep 08 14:02:14.424527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 585/1273] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:14 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:14.693302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9eeb0d65-c3d7-4211-98d2-864d5e70cd7d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:14.695404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9eeb0d65-c3d7-4211-98d2-864d5e70cd7d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:14.701494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9eeb0d65-c3d7-4211-98d2-864d5e70cd7d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:14.701992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9eeb0d65-c3d7-4211-98d2-864d5e70cd7d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:14.702464 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9eeb0d65-c3d7-4211-98d2-864d5e70cd7d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:14.703283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9eeb0d65-c3d7-4211-98d2-864d5e70cd7d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:14.703742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9eeb0d65-c3d7-4211-98d2-864d5e70cd7d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:14.704175 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9eeb0d65-c3d7-4211-98d2-864d5e70cd7d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:14.734959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9eeb0d65-c3d7-4211-98d2-864d5e70cd7d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045769 Sep 08 14:02:14.735527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 689/1274] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:14 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:14.946916 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-97f0b735-e5ec-4174-a841-785ba3ffb840 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:14.949035 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-97f0b735-e5ec-4174-a841-785ba3ffb840 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:14.955683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-97f0b735-e5ec-4174-a841-785ba3ffb840 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:14.956164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-97f0b735-e5ec-4174-a841-785ba3ffb840 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:14.956599 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-97f0b735-e5ec-4174-a841-785ba3ffb840 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:14.957286 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-97f0b735-e5ec-4174-a841-785ba3ffb840 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:14.957736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-97f0b735-e5ec-4174-a841-785ba3ffb840 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:14.958176 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-97f0b735-e5ec-4174-a841-785ba3ffb840 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:14.989521 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-97f0b735-e5ec-4174-a841-785ba3ffb840 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046889 Sep 08 14:02:14.990081 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 586/1275] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:14 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:15.068438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6b7b7b60-fdf4-4139-9b82-0db7ad263414 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:15.069043 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6b7b7b60-fdf4-4139-9b82-0db7ad263414 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:15.071466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:15.076537 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b7b7b60-fdf4-4139-9b82-0db7ad263414 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:15.077033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b7b7b60-fdf4-4139-9b82-0db7ad263414 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:15.077483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b7b7b60-fdf4-4139-9b82-0db7ad263414 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:15.078283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b7b7b60-fdf4-4139-9b82-0db7ad263414 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:15.078747 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b7b7b60-fdf4-4139-9b82-0db7ad263414 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:15.084978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b7b7b60-fdf4-4139-9b82-0db7ad263414 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:15.151562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6b7b7b60-fdf4-4139-9b82-0db7ad263414 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:15.162258 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6b7b7b60-fdf4-4139-9b82-0db7ad263414 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.097572 Sep 08 14:02:15.162836 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 690/1276] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:15 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:15.220025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8d6b61f9-5643-47fa-a988-6e47323c0da2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:15.222089 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8d6b61f9-5643-47fa-a988-6e47323c0da2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:15.229252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d6b61f9-5643-47fa-a988-6e47323c0da2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:15.229896 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d6b61f9-5643-47fa-a988-6e47323c0da2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:15.230268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d6b61f9-5643-47fa-a988-6e47323c0da2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:15.230989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d6b61f9-5643-47fa-a988-6e47323c0da2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:15.231405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d6b61f9-5643-47fa-a988-6e47323c0da2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:15.231824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d6b61f9-5643-47fa-a988-6e47323c0da2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:15.263859 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8d6b61f9-5643-47fa-a988-6e47323c0da2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048014 Sep 08 14:02:15.264446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 587/1277] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:15 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:15.448701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a80e7eb1-d4ff-4cd1-b9ec-af916abcaed6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:15.449317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a80e7eb1-d4ff-4cd1-b9ec-af916abcaed6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:15.455368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a80e7eb1-d4ff-4cd1-b9ec-af916abcaed6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:15.455858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a80e7eb1-d4ff-4cd1-b9ec-af916abcaed6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:15.456297 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a80e7eb1-d4ff-4cd1-b9ec-af916abcaed6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:15.457004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a80e7eb1-d4ff-4cd1-b9ec-af916abcaed6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:15.457455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a80e7eb1-d4ff-4cd1-b9ec-af916abcaed6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:15.457881 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a80e7eb1-d4ff-4cd1-b9ec-af916abcaed6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:15.602549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a80e7eb1-d4ff-4cd1-b9ec-af916abcaed6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:15.610929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a80e7eb1-d4ff-4cd1-b9ec-af916abcaed6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.166678 Sep 08 14:02:15.611508 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 691/1278] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:15 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:15.748233 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-eb2a31b0-054e-4fa0-95f1-c91731ee5c2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:15.750407 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-eb2a31b0-054e-4fa0-95f1-c91731ee5c2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:15.757914 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb2a31b0-054e-4fa0-95f1-c91731ee5c2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:15.758425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb2a31b0-054e-4fa0-95f1-c91731ee5c2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:15.758852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb2a31b0-054e-4fa0-95f1-c91731ee5c2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:15.759541 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb2a31b0-054e-4fa0-95f1-c91731ee5c2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:15.759979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb2a31b0-054e-4fa0-95f1-c91731ee5c2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:15.760394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb2a31b0-054e-4fa0-95f1-c91731ee5c2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:15.764210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:15.792265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-eb2a31b0-054e-4fa0-95f1-c91731ee5c2b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048090 Sep 08 14:02:15.792946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 588/1279] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:15 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:16.003649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-04e5303d-bb34-4a7e-9394-69b06baaa3f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:16.005809 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-04e5303d-bb34-4a7e-9394-69b06baaa3f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:16.012544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04e5303d-bb34-4a7e-9394-69b06baaa3f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:16.013031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04e5303d-bb34-4a7e-9394-69b06baaa3f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:16.013463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04e5303d-bb34-4a7e-9394-69b06baaa3f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:16.014161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04e5303d-bb34-4a7e-9394-69b06baaa3f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:16.014623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04e5303d-bb34-4a7e-9394-69b06baaa3f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:16.015042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04e5303d-bb34-4a7e-9394-69b06baaa3f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:16.039596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:16.044455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-04e5303d-bb34-4a7e-9394-69b06baaa3f5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045068 Sep 08 14:02:16.050940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 692/1280] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:15 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:16.191488 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e8323d55-b204-4895-9e69-df119fda9a76 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:16.192037 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e8323d55-b204-4895-9e69-df119fda9a76 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:16.199004 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8323d55-b204-4895-9e69-df119fda9a76 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:16.199480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8323d55-b204-4895-9e69-df119fda9a76 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:16.199908 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8323d55-b204-4895-9e69-df119fda9a76 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:16.200599 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8323d55-b204-4895-9e69-df119fda9a76 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:16.201041 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8323d55-b204-4895-9e69-df119fda9a76 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:16.201458 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e8323d55-b204-4895-9e69-df119fda9a76 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:16.264234 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e8323d55-b204-4895-9e69-df119fda9a76 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:16.272107 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e8323d55-b204-4895-9e69-df119fda9a76 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.085279 Sep 08 14:02:16.272671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 589/1281] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:16 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:16.276957 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4ebd1e1b-3da7-475f-b458-0bad210b3ca5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:16.279268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4ebd1e1b-3da7-475f-b458-0bad210b3ca5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:16.286095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ebd1e1b-3da7-475f-b458-0bad210b3ca5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:16.286618 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ebd1e1b-3da7-475f-b458-0bad210b3ca5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:16.287058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ebd1e1b-3da7-475f-b458-0bad210b3ca5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:16.287758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ebd1e1b-3da7-475f-b458-0bad210b3ca5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:16.288216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ebd1e1b-3da7-475f-b458-0bad210b3ca5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:16.289153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ebd1e1b-3da7-475f-b458-0bad210b3ca5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:16.321234 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4ebd1e1b-3da7-475f-b458-0bad210b3ca5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048741 Sep 08 14:02:16.321804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 693/1282] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:16 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:02:16.634400 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fade2d94-3e6b-4562-9aae-96c2f9395122 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:16.635004 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fade2d94-3e6b-4562-9aae-96c2f9395122 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:16.641100 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fade2d94-3e6b-4562-9aae-96c2f9395122 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:16.641586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fade2d94-3e6b-4562-9aae-96c2f9395122 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:16.642013 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fade2d94-3e6b-4562-9aae-96c2f9395122 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:16.642738 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fade2d94-3e6b-4562-9aae-96c2f9395122 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:16.643181 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fade2d94-3e6b-4562-9aae-96c2f9395122 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:16.643595 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fade2d94-3e6b-4562-9aae-96c2f9395122 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:16.779850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-fade2d94-3e6b-4562-9aae-96c2f9395122 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:16.787711 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fade2d94-3e6b-4562-9aae-96c2f9395122 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.157259 Sep 08 14:02:16.788287 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 590/1283] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:16 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:16.805411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-eb93a6f7-1dc3-430e-85cb-0c3a4da94e18 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:16.807972 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-eb93a6f7-1dc3-430e-85cb-0c3a4da94e18 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:16.814648 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb93a6f7-1dc3-430e-85cb-0c3a4da94e18 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:16.815151 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb93a6f7-1dc3-430e-85cb-0c3a4da94e18 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:16.815592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb93a6f7-1dc3-430e-85cb-0c3a4da94e18 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:16.816308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb93a6f7-1dc3-430e-85cb-0c3a4da94e18 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:16.816762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb93a6f7-1dc3-430e-85cb-0c3a4da94e18 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:16.817188 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb93a6f7-1dc3-430e-85cb-0c3a4da94e18 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:16.858429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-eb93a6f7-1dc3-430e-85cb-0c3a4da94e18 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057029 Sep 08 14:02:16.859251 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 694/1284] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:16 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:17.020951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-505cdfaf-5826-4d37-990e-e993b27ed5f2 req-f09ecd6e-5682-4c11-935a-d48b96b3fe01 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d1062abf-7525-48c5-a9cb-91da3855fd1b", "name": "network-vif-unplugged", "status": "completed", "tag": "2d04f3d2-99c9-4712-bc13-6ebdffcb949f"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:02:17.033448 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-505cdfaf-5826-4d37-990e-e993b27ed5f2 req-f09ecd6e-5682-4c11-935a-d48b96b3fe01 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:17.033923 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-505cdfaf-5826-4d37-990e-e993b27ed5f2 req-f09ecd6e-5682-4c11-935a-d48b96b3fe01 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:17.034383 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-505cdfaf-5826-4d37-990e-e993b27ed5f2 req-f09ecd6e-5682-4c11-935a-d48b96b3fe01 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:17.046120 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-505cdfaf-5826-4d37-990e-e993b27ed5f2 req-f09ecd6e-5682-4c11-935a-d48b96b3fe01 service nova] Creating event network-vif-unplugged:2d04f3d2-99c9-4712-bc13-6ebdffcb949f for instance d1062abf-7525-48c5-a9cb-91da3855fd1b on hv-cinder-85646 Sep 08 14:02:17.051505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-505cdfaf-5826-4d37-990e-e993b27ed5f2 req-f09ecd6e-5682-4c11-935a-d48b96b3fe01 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.034419 Sep 08 14:02:17.052117 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 591/1285] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:02:17 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:17.063958 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-811362a0-3a60-4fbe-9d22-ace75d6c4d2f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:17.066153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-811362a0-3a60-4fbe-9d22-ace75d6c4d2f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:17.075394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-811362a0-3a60-4fbe-9d22-ace75d6c4d2f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:17.076249 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-811362a0-3a60-4fbe-9d22-ace75d6c4d2f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:17.083631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-811362a0-3a60-4fbe-9d22-ace75d6c4d2f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:17.083631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-811362a0-3a60-4fbe-9d22-ace75d6c4d2f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:17.083631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-811362a0-3a60-4fbe-9d22-ace75d6c4d2f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:17.083631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-811362a0-3a60-4fbe-9d22-ace75d6c4d2f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:17.125983 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-811362a0-3a60-4fbe-9d22-ace75d6c4d2f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.068341 Sep 08 14:02:17.126606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 695/1286] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:17 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:17.297706 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-be87243d-e269-4b79-ad18-ca364bac7635 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:17.298640 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-be87243d-e269-4b79-ad18-ca364bac7635 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:17.305961 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be87243d-e269-4b79-ad18-ca364bac7635 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:17.306461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be87243d-e269-4b79-ad18-ca364bac7635 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:17.307030 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be87243d-e269-4b79-ad18-ca364bac7635 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:17.307741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be87243d-e269-4b79-ad18-ca364bac7635 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:17.308194 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be87243d-e269-4b79-ad18-ca364bac7635 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:17.308611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be87243d-e269-4b79-ad18-ca364bac7635 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:17.333038 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-45be3159-262b-4226-ae23-c05b36294c70 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:17.335309 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-45be3159-262b-4226-ae23-c05b36294c70 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:17.341842 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-45be3159-262b-4226-ae23-c05b36294c70 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:17.342370 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-45be3159-262b-4226-ae23-c05b36294c70 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:17.342807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-45be3159-262b-4226-ae23-c05b36294c70 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:17.343505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-45be3159-262b-4226-ae23-c05b36294c70 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:17.343988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-45be3159-262b-4226-ae23-c05b36294c70 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:17.344378 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-45be3159-262b-4226-ae23-c05b36294c70 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:17.376763 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-45be3159-262b-4226-ae23-c05b36294c70 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047652 Sep 08 14:02:17.377344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 696/1287] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:17 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:02:17.381419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-be87243d-e269-4b79-ad18-ca364bac7635 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:17.395775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-be87243d-e269-4b79-ad18-ca364bac7635 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.098261 Sep 08 14:02:17.395775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 592/1288] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:17 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:17.871755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2d1acfb1-f241-44a9-8b8a-0f963c6f3fdc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:17.873324 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b925a9e0-a55c-40ec-aac5-bdee7074c580 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:17.873936 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b925a9e0-a55c-40ec-aac5-bdee7074c580 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:17.874283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2d1acfb1-f241-44a9-8b8a-0f963c6f3fdc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:17.882733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b925a9e0-a55c-40ec-aac5-bdee7074c580 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:17.882733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b925a9e0-a55c-40ec-aac5-bdee7074c580 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:17.883050 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d1acfb1-f241-44a9-8b8a-0f963c6f3fdc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:17.883212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b925a9e0-a55c-40ec-aac5-bdee7074c580 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:17.883443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b925a9e0-a55c-40ec-aac5-bdee7074c580 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:17.883596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d1acfb1-f241-44a9-8b8a-0f963c6f3fdc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:17.883752 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b925a9e0-a55c-40ec-aac5-bdee7074c580 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:17.884262 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d1acfb1-f241-44a9-8b8a-0f963c6f3fdc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:17.884435 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b925a9e0-a55c-40ec-aac5-bdee7074c580 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:17.885161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d1acfb1-f241-44a9-8b8a-0f963c6f3fdc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:17.885766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d1acfb1-f241-44a9-8b8a-0f963c6f3fdc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:17.886380 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2d1acfb1-f241-44a9-8b8a-0f963c6f3fdc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:17.927440 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2d1acfb1-f241-44a9-8b8a-0f963c6f3fdc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060210 Sep 08 14:02:17.928017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 697/1289] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:17 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:18.035524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b925a9e0-a55c-40ec-aac5-bdee7074c580 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:18.045853 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b925a9e0-a55c-40ec-aac5-bdee7074c580 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.176653 Sep 08 14:02:18.046489 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 593/1290] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:17 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:18.243702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5aa7e4dd-96fd-48c2-b986-5677f00b6a0d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:18.246258 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5aa7e4dd-96fd-48c2-b986-5677f00b6a0d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:18.254170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aa7e4dd-96fd-48c2-b986-5677f00b6a0d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:18.254657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aa7e4dd-96fd-48c2-b986-5677f00b6a0d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:18.255096 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aa7e4dd-96fd-48c2-b986-5677f00b6a0d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:18.255796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aa7e4dd-96fd-48c2-b986-5677f00b6a0d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:18.256253 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aa7e4dd-96fd-48c2-b986-5677f00b6a0d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:18.256676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5aa7e4dd-96fd-48c2-b986-5677f00b6a0d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:18.287881 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5aa7e4dd-96fd-48c2-b986-5677f00b6a0d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.152699 Sep 08 14:02:18.288450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 698/1291] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:18 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:18.389726 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0e15a5be-d7de-46b4-b368-371bacd624cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:18.392106 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0e15a5be-d7de-46b4-b368-371bacd624cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:18.398756 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e15a5be-d7de-46b4-b368-371bacd624cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:18.399812 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e15a5be-d7de-46b4-b368-371bacd624cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:18.400246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e15a5be-d7de-46b4-b368-371bacd624cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:18.400942 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e15a5be-d7de-46b4-b368-371bacd624cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:18.401388 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e15a5be-d7de-46b4-b368-371bacd624cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:18.401803 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e15a5be-d7de-46b4-b368-371bacd624cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:18.417628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b71f9559-0b04-4d77-9b90-bd56dfb9b61f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:18.418208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b71f9559-0b04-4d77-9b90-bd56dfb9b61f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:18.424521 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b71f9559-0b04-4d77-9b90-bd56dfb9b61f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:18.425009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b71f9559-0b04-4d77-9b90-bd56dfb9b61f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:18.425445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b71f9559-0b04-4d77-9b90-bd56dfb9b61f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:18.426162 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b71f9559-0b04-4d77-9b90-bd56dfb9b61f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:18.426626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b71f9559-0b04-4d77-9b90-bd56dfb9b61f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:18.427050 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b71f9559-0b04-4d77-9b90-bd56dfb9b61f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:18.432685 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0e15a5be-d7de-46b4-b368-371bacd624cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046797 Sep 08 14:02:18.434665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 594/1292] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:18 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:18.492236 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b71f9559-0b04-4d77-9b90-bd56dfb9b61f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:18.500780 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b71f9559-0b04-4d77-9b90-bd56dfb9b61f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.087233 Sep 08 14:02:18.501354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 699/1293] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:18 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:18.941698 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f59cc108-48e4-43a2-8387-ba56648c80a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:18.944037 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f59cc108-48e4-43a2-8387-ba56648c80a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:18.951374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f59cc108-48e4-43a2-8387-ba56648c80a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:18.951903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f59cc108-48e4-43a2-8387-ba56648c80a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:18.952381 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f59cc108-48e4-43a2-8387-ba56648c80a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:18.953087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f59cc108-48e4-43a2-8387-ba56648c80a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:18.953569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f59cc108-48e4-43a2-8387-ba56648c80a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:18.954028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f59cc108-48e4-43a2-8387-ba56648c80a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:18.989549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f59cc108-48e4-43a2-8387-ba56648c80a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051939 Sep 08 14:02:18.990210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 595/1294] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:18 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:19.069458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f4ddbd9d-6ae7-4697-9a41-00cb2415f277 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:19.070274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f4ddbd9d-6ae7-4697-9a41-00cb2415f277 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:19.078819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f4ddbd9d-6ae7-4697-9a41-00cb2415f277 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:19.079378 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f4ddbd9d-6ae7-4697-9a41-00cb2415f277 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:19.079827 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f4ddbd9d-6ae7-4697-9a41-00cb2415f277 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:19.080553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f4ddbd9d-6ae7-4697-9a41-00cb2415f277 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:19.081036 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f4ddbd9d-6ae7-4697-9a41-00cb2415f277 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:19.081447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f4ddbd9d-6ae7-4697-9a41-00cb2415f277 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:19.225156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f4ddbd9d-6ae7-4697-9a41-00cb2415f277 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:19.233912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f4ddbd9d-6ae7-4697-9a41-00cb2415f277 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.168437 Sep 08 14:02:19.234524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 700/1295] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:19 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:19.301743 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d75e7399-4069-4c24-80da-8ed1513d11b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:19.303980 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d75e7399-4069-4c24-80da-8ed1513d11b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:19.311660 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d75e7399-4069-4c24-80da-8ed1513d11b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:19.312148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d75e7399-4069-4c24-80da-8ed1513d11b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:19.312590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d75e7399-4069-4c24-80da-8ed1513d11b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:19.313348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d75e7399-4069-4c24-80da-8ed1513d11b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:19.313743 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d75e7399-4069-4c24-80da-8ed1513d11b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:19.314529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d75e7399-4069-4c24-80da-8ed1513d11b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:19.345271 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d75e7399-4069-4c24-80da-8ed1513d11b5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047581 Sep 08 14:02:19.345845 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 596/1296] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:19 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:19.450855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-81eacc50-ebd9-4e29-80dc-f4d17e564b68 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:19.453078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-81eacc50-ebd9-4e29-80dc-f4d17e564b68 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:19.459517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81eacc50-ebd9-4e29-80dc-f4d17e564b68 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:19.460007 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81eacc50-ebd9-4e29-80dc-f4d17e564b68 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:19.460448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81eacc50-ebd9-4e29-80dc-f4d17e564b68 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:19.461150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81eacc50-ebd9-4e29-80dc-f4d17e564b68 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:19.461783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81eacc50-ebd9-4e29-80dc-f4d17e564b68 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:19.462177 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81eacc50-ebd9-4e29-80dc-f4d17e564b68 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:19.491822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-81eacc50-ebd9-4e29-80dc-f4d17e564b68 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044885 Sep 08 14:02:19.492308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 701/1297] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:19 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:19.525269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9d948338-812d-4cd8-8130-6ef91e1ac627 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:19.525878 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9d948338-812d-4cd8-8130-6ef91e1ac627 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:19.532521 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9d948338-812d-4cd8-8130-6ef91e1ac627 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:19.533007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9d948338-812d-4cd8-8130-6ef91e1ac627 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:19.533446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9d948338-812d-4cd8-8130-6ef91e1ac627 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:19.534158 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9d948338-812d-4cd8-8130-6ef91e1ac627 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:19.534621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9d948338-812d-4cd8-8130-6ef91e1ac627 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:19.535044 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9d948338-812d-4cd8-8130-6ef91e1ac627 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:19.601262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9d948338-812d-4cd8-8130-6ef91e1ac627 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:19.609952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9d948338-812d-4cd8-8130-6ef91e1ac627 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.089249 Sep 08 14:02:19.610568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 597/1298] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:19 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:20.006238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-64729ff6-5a76-4949-9d35-3169b8b69660 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:20.010094 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-64729ff6-5a76-4949-9d35-3169b8b69660 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:20.019766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-64729ff6-5a76-4949-9d35-3169b8b69660 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:20.020612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-64729ff6-5a76-4949-9d35-3169b8b69660 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:20.021321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-64729ff6-5a76-4949-9d35-3169b8b69660 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:20.022131 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-64729ff6-5a76-4949-9d35-3169b8b69660 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:20.022643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-64729ff6-5a76-4949-9d35-3169b8b69660 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:20.023081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-64729ff6-5a76-4949-9d35-3169b8b69660 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:20.057561 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-64729ff6-5a76-4949-9d35-3169b8b69660 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058491 Sep 08 14:02:20.058180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 702/1299] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:19 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:20.259472 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-84302f0b-327f-4aff-b759-83952b955ec8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:20.260016 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-84302f0b-327f-4aff-b759-83952b955ec8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:20.266358 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84302f0b-327f-4aff-b759-83952b955ec8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:20.266847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84302f0b-327f-4aff-b759-83952b955ec8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:20.267286 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84302f0b-327f-4aff-b759-83952b955ec8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:20.267992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84302f0b-327f-4aff-b759-83952b955ec8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:20.268448 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84302f0b-327f-4aff-b759-83952b955ec8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:20.268874 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84302f0b-327f-4aff-b759-83952b955ec8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:20.372000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9c8d0617-2ef9-433f-923a-3e87b5e49225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:20.374577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9c8d0617-2ef9-433f-923a-3e87b5e49225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:20.384846 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9c8d0617-2ef9-433f-923a-3e87b5e49225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:20.385583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9c8d0617-2ef9-433f-923a-3e87b5e49225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:20.386069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9c8d0617-2ef9-433f-923a-3e87b5e49225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:20.387019 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9c8d0617-2ef9-433f-923a-3e87b5e49225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:20.387511 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9c8d0617-2ef9-433f-923a-3e87b5e49225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:20.387997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9c8d0617-2ef9-433f-923a-3e87b5e49225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:20.421491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9c8d0617-2ef9-433f-923a-3e87b5e49225 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066913 Sep 08 14:02:20.422079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 703/1300] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:20 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:20.447446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-84302f0b-327f-4aff-b759-83952b955ec8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:20.458326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-84302f0b-327f-4aff-b759-83952b955ec8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.203941 Sep 08 14:02:20.458921 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 598/1301] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:20 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:20.506912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3614553c-eb70-442a-acde-df55fccba016 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:20.507624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3614553c-eb70-442a-acde-df55fccba016 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:20.515403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3614553c-eb70-442a-acde-df55fccba016 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:20.515816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3614553c-eb70-442a-acde-df55fccba016 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:20.516248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3614553c-eb70-442a-acde-df55fccba016 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:20.517006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3614553c-eb70-442a-acde-df55fccba016 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:20.517442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3614553c-eb70-442a-acde-df55fccba016 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:20.518538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3614553c-eb70-442a-acde-df55fccba016 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:20.556364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3614553c-eb70-442a-acde-df55fccba016 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055296 Sep 08 14:02:20.556943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 704/1302] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:20 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:20.633829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-aafac291-dd88-4edc-a650-588dfb19ed1b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:20.634654 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-aafac291-dd88-4edc-a650-588dfb19ed1b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:20.641501 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aafac291-dd88-4edc-a650-588dfb19ed1b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:20.641897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aafac291-dd88-4edc-a650-588dfb19ed1b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:20.642360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aafac291-dd88-4edc-a650-588dfb19ed1b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:20.643057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aafac291-dd88-4edc-a650-588dfb19ed1b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:20.643503 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aafac291-dd88-4edc-a650-588dfb19ed1b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:20.643917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aafac291-dd88-4edc-a650-588dfb19ed1b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:20.711457 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-aafac291-dd88-4edc-a650-588dfb19ed1b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:20.720500 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-aafac291-dd88-4edc-a650-588dfb19ed1b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.090576 Sep 08 14:02:20.721088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 599/1303] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:20 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:21.070466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a2b2c050-8dd7-4a3c-a757-2d666a9ffc31 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:21.072949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a2b2c050-8dd7-4a3c-a757-2d666a9ffc31 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:21.079399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a2b2c050-8dd7-4a3c-a757-2d666a9ffc31 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:21.079888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a2b2c050-8dd7-4a3c-a757-2d666a9ffc31 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:21.080333 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a2b2c050-8dd7-4a3c-a757-2d666a9ffc31 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:21.081060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a2b2c050-8dd7-4a3c-a757-2d666a9ffc31 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:21.081516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a2b2c050-8dd7-4a3c-a757-2d666a9ffc31 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:21.081943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a2b2c050-8dd7-4a3c-a757-2d666a9ffc31 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:21.113844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a2b2c050-8dd7-4a3c-a757-2d666a9ffc31 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047393 Sep 08 14:02:21.114438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 705/1304] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:21 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:21.435079 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-dedfb84a-7866-48ac-91b4-e239b89f7a53 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:21.437411 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-dedfb84a-7866-48ac-91b4-e239b89f7a53 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:21.443576 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dedfb84a-7866-48ac-91b4-e239b89f7a53 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:21.444052 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dedfb84a-7866-48ac-91b4-e239b89f7a53 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:21.444480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dedfb84a-7866-48ac-91b4-e239b89f7a53 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:21.445171 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dedfb84a-7866-48ac-91b4-e239b89f7a53 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:21.446248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dedfb84a-7866-48ac-91b4-e239b89f7a53 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:21.446674 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dedfb84a-7866-48ac-91b4-e239b89f7a53 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:21.476517 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-dedfb84a-7866-48ac-91b4-e239b89f7a53 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046042 Sep 08 14:02:21.477119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 600/1305] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:21 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:21.482929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7e68f9a7-4de9-43af-ae93-58287d266c81 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:21.483741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7e68f9a7-4de9-43af-ae93-58287d266c81 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:21.489815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7e68f9a7-4de9-43af-ae93-58287d266c81 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:21.490360 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7e68f9a7-4de9-43af-ae93-58287d266c81 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:21.490794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7e68f9a7-4de9-43af-ae93-58287d266c81 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:21.491479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7e68f9a7-4de9-43af-ae93-58287d266c81 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:21.491924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7e68f9a7-4de9-43af-ae93-58287d266c81 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:21.492341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7e68f9a7-4de9-43af-ae93-58287d266c81 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:21.573653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-89238d8c-2d00-4602-a95e-64f87b11b6fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:21.577268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-89238d8c-2d00-4602-a95e-64f87b11b6fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:21.588297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89238d8c-2d00-4602-a95e-64f87b11b6fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:21.589357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89238d8c-2d00-4602-a95e-64f87b11b6fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:21.590000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89238d8c-2d00-4602-a95e-64f87b11b6fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:21.591445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89238d8c-2d00-4602-a95e-64f87b11b6fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:21.592267 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89238d8c-2d00-4602-a95e-64f87b11b6fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:21.593041 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89238d8c-2d00-4602-a95e-64f87b11b6fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:21.632491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7e68f9a7-4de9-43af-ae93-58287d266c81 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:21.636198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-89238d8c-2d00-4602-a95e-64f87b11b6fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.070715 Sep 08 14:02:21.637080 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 601/1306] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:21 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:21.641840 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7e68f9a7-4de9-43af-ae93-58287d266c81 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.162909 Sep 08 14:02:21.642469 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 706/1307] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:21 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:21.744976 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bd981bfd-8a10-4752-a4ea-2225b6e17571 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:21.745491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bd981bfd-8a10-4752-a4ea-2225b6e17571 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:21.752331 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd981bfd-8a10-4752-a4ea-2225b6e17571 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:21.753074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd981bfd-8a10-4752-a4ea-2225b6e17571 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:21.753500 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd981bfd-8a10-4752-a4ea-2225b6e17571 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:21.754225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd981bfd-8a10-4752-a4ea-2225b6e17571 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:21.754672 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd981bfd-8a10-4752-a4ea-2225b6e17571 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:21.755139 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd981bfd-8a10-4752-a4ea-2225b6e17571 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:21.823786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-bd981bfd-8a10-4752-a4ea-2225b6e17571 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:21.831865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bd981bfd-8a10-4752-a4ea-2225b6e17571 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.091512 Sep 08 14:02:21.832441 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 602/1308] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:21 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:22.127188 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-09343a03-fbb3-4c7f-8709-eafece64f1bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:22.129659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-09343a03-fbb3-4c7f-8709-eafece64f1bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:22.135925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-09343a03-fbb3-4c7f-8709-eafece64f1bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:22.136409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-09343a03-fbb3-4c7f-8709-eafece64f1bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:22.136870 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-09343a03-fbb3-4c7f-8709-eafece64f1bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:22.137567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-09343a03-fbb3-4c7f-8709-eafece64f1bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:22.138019 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-09343a03-fbb3-4c7f-8709-eafece64f1bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:22.138480 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-09343a03-fbb3-4c7f-8709-eafece64f1bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:22.168475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-09343a03-fbb3-4c7f-8709-eafece64f1bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045215 Sep 08 14:02:22.169031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 707/1309] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:22 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:22.493453 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c5158889-8853-4a50-ac06-09b5cb3bfafa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:22.497085 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c5158889-8853-4a50-ac06-09b5cb3bfafa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:22.506528 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5158889-8853-4a50-ac06-09b5cb3bfafa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:22.507018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5158889-8853-4a50-ac06-09b5cb3bfafa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:22.507455 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5158889-8853-4a50-ac06-09b5cb3bfafa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:22.508159 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5158889-8853-4a50-ac06-09b5cb3bfafa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:22.508735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5158889-8853-4a50-ac06-09b5cb3bfafa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:22.509347 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5158889-8853-4a50-ac06-09b5cb3bfafa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:22.538246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c5158889-8853-4a50-ac06-09b5cb3bfafa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052644 Sep 08 14:02:22.538807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 603/1310] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:22 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:22.649659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-21167525-7b8f-4c95-ba92-2cf617381e29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:22.651880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-21167525-7b8f-4c95-ba92-2cf617381e29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:22.658301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-21167525-7b8f-4c95-ba92-2cf617381e29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:22.658788 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-21167525-7b8f-4c95-ba92-2cf617381e29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:22.659227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-21167525-7b8f-4c95-ba92-2cf617381e29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:22.659916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-21167525-7b8f-4c95-ba92-2cf617381e29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:22.661033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-21167525-7b8f-4c95-ba92-2cf617381e29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:22.661672 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-21167525-7b8f-4c95-ba92-2cf617381e29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:22.667199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bebc0ff1-4bd0-43e5-9cac-db77d941bc93 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:22.667813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bebc0ff1-4bd0-43e5-9cac-db77d941bc93 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:22.673794 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bebc0ff1-4bd0-43e5-9cac-db77d941bc93 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:22.674314 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bebc0ff1-4bd0-43e5-9cac-db77d941bc93 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:22.674754 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bebc0ff1-4bd0-43e5-9cac-db77d941bc93 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:22.675452 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bebc0ff1-4bd0-43e5-9cac-db77d941bc93 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:22.675930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bebc0ff1-4bd0-43e5-9cac-db77d941bc93 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:22.676333 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bebc0ff1-4bd0-43e5-9cac-db77d941bc93 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:22.692690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-21167525-7b8f-4c95-ba92-2cf617381e29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047866 Sep 08 14:02:22.693256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 708/1311] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:22 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:22.808586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-bebc0ff1-4bd0-43e5-9cac-db77d941bc93 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:22.816377 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bebc0ff1-4bd0-43e5-9cac-db77d941bc93 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.153696 Sep 08 14:02:22.816946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 604/1312] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:22 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:22.856209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3244db08-595b-4892-aea8-6db1edeab253 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:22.856816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3244db08-595b-4892-aea8-6db1edeab253 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:22.862774 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3244db08-595b-4892-aea8-6db1edeab253 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:22.863259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3244db08-595b-4892-aea8-6db1edeab253 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:22.863700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3244db08-595b-4892-aea8-6db1edeab253 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:22.864395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3244db08-595b-4892-aea8-6db1edeab253 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:22.864849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3244db08-595b-4892-aea8-6db1edeab253 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:22.865275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3244db08-595b-4892-aea8-6db1edeab253 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:22.880452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:22.927487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3244db08-595b-4892-aea8-6db1edeab253 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:22.935117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3244db08-595b-4892-aea8-6db1edeab253 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.083845 Sep 08 14:02:22.935683 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 709/1313] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:22 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:23.181098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-efa1f6d2-a23f-4765-9163-6597bf83d8b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:23.183258 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-efa1f6d2-a23f-4765-9163-6597bf83d8b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:23.190016 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-efa1f6d2-a23f-4765-9163-6597bf83d8b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:23.190529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-efa1f6d2-a23f-4765-9163-6597bf83d8b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:23.190954 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-efa1f6d2-a23f-4765-9163-6597bf83d8b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:23.191681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-efa1f6d2-a23f-4765-9163-6597bf83d8b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:23.192129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-efa1f6d2-a23f-4765-9163-6597bf83d8b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:23.192547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-efa1f6d2-a23f-4765-9163-6597bf83d8b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:23.210421 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:23.224409 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-efa1f6d2-a23f-4765-9163-6597bf83d8b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047396 Sep 08 14:02:23.227172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 605/1314] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:23 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:23.552038 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e4ef0979-6fe6-485f-8e7f-7f91168637bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:23.554281 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e4ef0979-6fe6-485f-8e7f-7f91168637bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:23.561330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e4ef0979-6fe6-485f-8e7f-7f91168637bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:23.561810 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e4ef0979-6fe6-485f-8e7f-7f91168637bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:23.562287 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e4ef0979-6fe6-485f-8e7f-7f91168637bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:23.562960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e4ef0979-6fe6-485f-8e7f-7f91168637bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:23.563404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e4ef0979-6fe6-485f-8e7f-7f91168637bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:23.563823 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e4ef0979-6fe6-485f-8e7f-7f91168637bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:23.594917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e4ef0979-6fe6-485f-8e7f-7f91168637bd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046841 Sep 08 14:02:23.595488 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 710/1315] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:23 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:23.707973 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e7b44ca2-956e-4b05-9fff-88d156ec74ec tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:23.710163 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e7b44ca2-956e-4b05-9fff-88d156ec74ec tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:23.716846 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7b44ca2-956e-4b05-9fff-88d156ec74ec tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:23.717333 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7b44ca2-956e-4b05-9fff-88d156ec74ec tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:23.717886 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7b44ca2-956e-4b05-9fff-88d156ec74ec tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:23.718627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7b44ca2-956e-4b05-9fff-88d156ec74ec tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:23.719082 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7b44ca2-956e-4b05-9fff-88d156ec74ec tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:23.719577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7b44ca2-956e-4b05-9fff-88d156ec74ec tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:23.752197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e7b44ca2-956e-4b05-9fff-88d156ec74ec tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050017 Sep 08 14:02:23.752970 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 606/1316] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:23 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:23.840450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a971a7cc-edb0-4cb2-a237-97ab467af972 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:23.841080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a971a7cc-edb0-4cb2-a237-97ab467af972 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:23.847767 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a971a7cc-edb0-4cb2-a237-97ab467af972 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:23.848257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a971a7cc-edb0-4cb2-a237-97ab467af972 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:23.848696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a971a7cc-edb0-4cb2-a237-97ab467af972 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:23.849393 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a971a7cc-edb0-4cb2-a237-97ab467af972 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:23.849850 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a971a7cc-edb0-4cb2-a237-97ab467af972 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:23.850310 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a971a7cc-edb0-4cb2-a237-97ab467af972 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:23.958070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9744d860-da94-4fbc-a4d2-f01a701132dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:23.958716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9744d860-da94-4fbc-a4d2-f01a701132dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:23.966939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9744d860-da94-4fbc-a4d2-f01a701132dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:23.967430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9744d860-da94-4fbc-a4d2-f01a701132dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:23.967865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9744d860-da94-4fbc-a4d2-f01a701132dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:23.968568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9744d860-da94-4fbc-a4d2-f01a701132dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:23.969022 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9744d860-da94-4fbc-a4d2-f01a701132dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:23.969445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9744d860-da94-4fbc-a4d2-f01a701132dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:24.019352 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a971a7cc-edb0-4cb2-a237-97ab467af972 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:24.029705 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a971a7cc-edb0-4cb2-a237-97ab467af972 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.193211 Sep 08 14:02:24.033493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 711/1317] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:23 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:24.037606 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9744d860-da94-4fbc-a4d2-f01a701132dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:24.046811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9744d860-da94-4fbc-a4d2-f01a701132dc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.092476 Sep 08 14:02:24.047433 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 607/1318] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:23 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:24.241000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e84cc684-d8ec-44c9-8c0e-2d46c706e813 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:24.243526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e84cc684-d8ec-44c9-8c0e-2d46c706e813 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:24.250049 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e84cc684-d8ec-44c9-8c0e-2d46c706e813 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:24.250565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e84cc684-d8ec-44c9-8c0e-2d46c706e813 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:24.251002 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e84cc684-d8ec-44c9-8c0e-2d46c706e813 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:24.251719 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e84cc684-d8ec-44c9-8c0e-2d46c706e813 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:24.252169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e84cc684-d8ec-44c9-8c0e-2d46c706e813 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:24.252864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e84cc684-d8ec-44c9-8c0e-2d46c706e813 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:24.295415 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e84cc684-d8ec-44c9-8c0e-2d46c706e813 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060720 Sep 08 14:02:24.295894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 712/1319] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:24 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:24.608415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1c7bc163-f87d-4e01-83cc-6a19412519df tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:24.610599 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1c7bc163-f87d-4e01-83cc-6a19412519df tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:24.620524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c7bc163-f87d-4e01-83cc-6a19412519df tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:24.620524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c7bc163-f87d-4e01-83cc-6a19412519df tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:24.620524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c7bc163-f87d-4e01-83cc-6a19412519df tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:24.625071 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c7bc163-f87d-4e01-83cc-6a19412519df tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:24.625071 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c7bc163-f87d-4e01-83cc-6a19412519df tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:24.625071 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c7bc163-f87d-4e01-83cc-6a19412519df tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:24.653996 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1c7bc163-f87d-4e01-83cc-6a19412519df tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049372 Sep 08 14:02:24.710869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 608/1320] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:24 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:24.765092 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-24e6c3c2-d071-46d7-9abb-83041546d5e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:24.767328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-24e6c3c2-d071-46d7-9abb-83041546d5e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:24.774304 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-24e6c3c2-d071-46d7-9abb-83041546d5e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:24.774781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-24e6c3c2-d071-46d7-9abb-83041546d5e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:24.775211 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-24e6c3c2-d071-46d7-9abb-83041546d5e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:24.775901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-24e6c3c2-d071-46d7-9abb-83041546d5e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:24.776346 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-24e6c3c2-d071-46d7-9abb-83041546d5e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:24.776797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-24e6c3c2-d071-46d7-9abb-83041546d5e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:24.807807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-24e6c3c2-d071-46d7-9abb-83041546d5e9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046758 Sep 08 14:02:24.808377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 713/1321] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:24 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:25.053787 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ddb8bb81-951c-43f5-b809-e0d388b9aef3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:25.054385 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ddb8bb81-951c-43f5-b809-e0d388b9aef3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:25.061031 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ddb8bb81-951c-43f5-b809-e0d388b9aef3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:25.061516 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ddb8bb81-951c-43f5-b809-e0d388b9aef3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:25.061953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ddb8bb81-951c-43f5-b809-e0d388b9aef3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:25.062679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ddb8bb81-951c-43f5-b809-e0d388b9aef3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:25.063132 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ddb8bb81-951c-43f5-b809-e0d388b9aef3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:25.063589 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ddb8bb81-951c-43f5-b809-e0d388b9aef3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:25.070357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-182a9598-fa2b-41b8-a25c-6a9d2beee302 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:25.071707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-182a9598-fa2b-41b8-a25c-6a9d2beee302 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:25.078257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-182a9598-fa2b-41b8-a25c-6a9d2beee302 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:25.078754 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-182a9598-fa2b-41b8-a25c-6a9d2beee302 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:25.079196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-182a9598-fa2b-41b8-a25c-6a9d2beee302 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:25.079907 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-182a9598-fa2b-41b8-a25c-6a9d2beee302 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:25.080363 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-182a9598-fa2b-41b8-a25c-6a9d2beee302 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:25.080799 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-182a9598-fa2b-41b8-a25c-6a9d2beee302 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:25.162760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-182a9598-fa2b-41b8-a25c-6a9d2beee302 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:25.176346 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-182a9598-fa2b-41b8-a25c-6a9d2beee302 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.109672 Sep 08 14:02:25.177388 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 714/1322] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:25 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:25.233184 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ddb8bb81-951c-43f5-b809-e0d388b9aef3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:25.242698 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ddb8bb81-951c-43f5-b809-e0d388b9aef3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.192689 Sep 08 14:02:25.243471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 609/1323] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:25 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:25.311145 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-58763e25-92ca-4888-b912-2367e0d1c07c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:25.315476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-58763e25-92ca-4888-b912-2367e0d1c07c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:25.324126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58763e25-92ca-4888-b912-2367e0d1c07c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:25.324620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58763e25-92ca-4888-b912-2367e0d1c07c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:25.325059 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58763e25-92ca-4888-b912-2367e0d1c07c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:25.325763 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58763e25-92ca-4888-b912-2367e0d1c07c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:25.326251 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58763e25-92ca-4888-b912-2367e0d1c07c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:25.326683 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58763e25-92ca-4888-b912-2367e0d1c07c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:25.359164 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-58763e25-92ca-4888-b912-2367e0d1c07c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055493 Sep 08 14:02:25.359902 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 715/1324] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:25 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:25.667990 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8d379ba7-555c-4d83-8047-10fc14702dfa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:25.670266 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8d379ba7-555c-4d83-8047-10fc14702dfa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:25.676242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d379ba7-555c-4d83-8047-10fc14702dfa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:25.676723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d379ba7-555c-4d83-8047-10fc14702dfa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:25.677157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d379ba7-555c-4d83-8047-10fc14702dfa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:25.677842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d379ba7-555c-4d83-8047-10fc14702dfa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:25.678351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d379ba7-555c-4d83-8047-10fc14702dfa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:25.678814 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d379ba7-555c-4d83-8047-10fc14702dfa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:25.708951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8d379ba7-555c-4d83-8047-10fc14702dfa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045538 Sep 08 14:02:25.709500 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 610/1325] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:25 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:25.820593 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f0247741-0df4-4c5d-9863-055a7fe8b323 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:25.822782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f0247741-0df4-4c5d-9863-055a7fe8b323 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:25.829625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0247741-0df4-4c5d-9863-055a7fe8b323 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:25.830224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0247741-0df4-4c5d-9863-055a7fe8b323 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:25.830740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0247741-0df4-4c5d-9863-055a7fe8b323 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:25.831651 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0247741-0df4-4c5d-9863-055a7fe8b323 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:25.832108 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0247741-0df4-4c5d-9863-055a7fe8b323 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:25.832544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0247741-0df4-4c5d-9863-055a7fe8b323 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:25.864002 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f0247741-0df4-4c5d-9863-055a7fe8b323 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048038 Sep 08 14:02:25.864570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 716/1326] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:25 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:26.201660 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ccc9c357-7e8c-4b76-9f27-bfcf348a8558 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:26.202312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ccc9c357-7e8c-4b76-9f27-bfcf348a8558 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:26.209001 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccc9c357-7e8c-4b76-9f27-bfcf348a8558 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:26.209484 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccc9c357-7e8c-4b76-9f27-bfcf348a8558 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:26.209913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccc9c357-7e8c-4b76-9f27-bfcf348a8558 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:26.210657 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccc9c357-7e8c-4b76-9f27-bfcf348a8558 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:26.211097 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccc9c357-7e8c-4b76-9f27-bfcf348a8558 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:26.211514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccc9c357-7e8c-4b76-9f27-bfcf348a8558 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:26.267740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-495f0b44-fa5a-4132-ad99-1529ce0b9f34 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:26.268386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-495f0b44-fa5a-4132-ad99-1529ce0b9f34 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:26.275934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-495f0b44-fa5a-4132-ad99-1529ce0b9f34 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:26.276430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-495f0b44-fa5a-4132-ad99-1529ce0b9f34 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:26.276673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ccc9c357-7e8c-4b76-9f27-bfcf348a8558 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:26.276888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-495f0b44-fa5a-4132-ad99-1529ce0b9f34 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:26.277566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-495f0b44-fa5a-4132-ad99-1529ce0b9f34 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:26.278029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-495f0b44-fa5a-4132-ad99-1529ce0b9f34 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:26.278489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-495f0b44-fa5a-4132-ad99-1529ce0b9f34 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:26.284684 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ccc9c357-7e8c-4b76-9f27-bfcf348a8558 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.088057 Sep 08 14:02:26.290205 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 611/1327] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:26 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:26.372867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-522bdb24-d50c-44ee-afa4-0f3ed3869e97 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:26.375068 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-522bdb24-d50c-44ee-afa4-0f3ed3869e97 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:26.383199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-522bdb24-d50c-44ee-afa4-0f3ed3869e97 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:26.383947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-522bdb24-d50c-44ee-afa4-0f3ed3869e97 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:26.384379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-522bdb24-d50c-44ee-afa4-0f3ed3869e97 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:26.385082 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-522bdb24-d50c-44ee-afa4-0f3ed3869e97 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:26.385530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-522bdb24-d50c-44ee-afa4-0f3ed3869e97 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:26.385947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-522bdb24-d50c-44ee-afa4-0f3ed3869e97 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:26.418063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-522bdb24-d50c-44ee-afa4-0f3ed3869e97 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049433 Sep 08 14:02:26.418862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 612/1328] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:26 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:26.420799 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-495f0b44-fa5a-4132-ad99-1529ce0b9f34 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:26.429694 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-495f0b44-fa5a-4132-ad99-1529ce0b9f34 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.165441 Sep 08 14:02:26.430322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 717/1329] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:26 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:26.725006 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fad73b6f-42bf-4e77-846e-1aac52af5277 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:26.727208 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fad73b6f-42bf-4e77-846e-1aac52af5277 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:26.733592 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fad73b6f-42bf-4e77-846e-1aac52af5277 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:26.734086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fad73b6f-42bf-4e77-846e-1aac52af5277 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:26.734566 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fad73b6f-42bf-4e77-846e-1aac52af5277 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:26.735350 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fad73b6f-42bf-4e77-846e-1aac52af5277 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:26.735805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fad73b6f-42bf-4e77-846e-1aac52af5277 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:26.736320 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fad73b6f-42bf-4e77-846e-1aac52af5277 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:26.773918 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fad73b6f-42bf-4e77-846e-1aac52af5277 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052188 Sep 08 14:02:26.774663 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 613/1330] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:26 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:26.877754 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0e3268ea-e05b-4eca-aab8-0bc118e7951d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:26.879936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0e3268ea-e05b-4eca-aab8-0bc118e7951d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:26.890682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e3268ea-e05b-4eca-aab8-0bc118e7951d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:26.890682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e3268ea-e05b-4eca-aab8-0bc118e7951d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:26.891120 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e3268ea-e05b-4eca-aab8-0bc118e7951d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:26.892334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e3268ea-e05b-4eca-aab8-0bc118e7951d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:26.893162 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e3268ea-e05b-4eca-aab8-0bc118e7951d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:26.893954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e3268ea-e05b-4eca-aab8-0bc118e7951d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:26.933086 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0e3268ea-e05b-4eca-aab8-0bc118e7951d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.059611 Sep 08 14:02:26.933659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 718/1331] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:26 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:02:27.311941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-03ff99f0-b211-445d-a06c-79d2428538c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:27.312525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-03ff99f0-b211-445d-a06c-79d2428538c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:27.319025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-03ff99f0-b211-445d-a06c-79d2428538c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:27.320100 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-03ff99f0-b211-445d-a06c-79d2428538c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:27.320568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-03ff99f0-b211-445d-a06c-79d2428538c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:27.321266 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-03ff99f0-b211-445d-a06c-79d2428538c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:27.321782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-03ff99f0-b211-445d-a06c-79d2428538c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:27.322314 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-03ff99f0-b211-445d-a06c-79d2428538c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:27.388642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-03ff99f0-b211-445d-a06c-79d2428538c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:27.396915 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-03ff99f0-b211-445d-a06c-79d2428538c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.089569 Sep 08 14:02:27.397941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 614/1332] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:27 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:27.431858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6f0bea40-d755-47a1-b2fd-892fca71d9fe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:27.433916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6f0bea40-d755-47a1-b2fd-892fca71d9fe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:27.440237 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f0bea40-d755-47a1-b2fd-892fca71d9fe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:27.440724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f0bea40-d755-47a1-b2fd-892fca71d9fe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:27.441165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f0bea40-d755-47a1-b2fd-892fca71d9fe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:27.441866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f0bea40-d755-47a1-b2fd-892fca71d9fe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:27.442354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f0bea40-d755-47a1-b2fd-892fca71d9fe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:27.442780 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f0bea40-d755-47a1-b2fd-892fca71d9fe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:27.459433 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b6512e09-ebce-4697-a504-c6db2860fdc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:27.460082 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b6512e09-ebce-4697-a504-c6db2860fdc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:27.466193 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b6512e09-ebce-4697-a504-c6db2860fdc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:27.466701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b6512e09-ebce-4697-a504-c6db2860fdc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:27.467134 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b6512e09-ebce-4697-a504-c6db2860fdc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:27.467844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b6512e09-ebce-4697-a504-c6db2860fdc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:27.468297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b6512e09-ebce-4697-a504-c6db2860fdc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:27.468712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b6512e09-ebce-4697-a504-c6db2860fdc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:27.474627 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6f0bea40-d755-47a1-b2fd-892fca71d9fe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047639 Sep 08 14:02:27.475202 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 719/1333] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:27 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:27.634014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b6512e09-ebce-4697-a504-c6db2860fdc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:27.643843 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b6512e09-ebce-4697-a504-c6db2860fdc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.188601 Sep 08 14:02:27.644437 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 615/1334] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:27 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:27.786665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-dec17a8a-1420-4584-a85b-768649d3252e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:27.789529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-dec17a8a-1420-4584-a85b-768649d3252e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:27.795740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dec17a8a-1420-4584-a85b-768649d3252e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:27.796224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dec17a8a-1420-4584-a85b-768649d3252e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:27.796659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dec17a8a-1420-4584-a85b-768649d3252e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:27.797357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dec17a8a-1420-4584-a85b-768649d3252e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:27.797810 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dec17a8a-1420-4584-a85b-768649d3252e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:27.798271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dec17a8a-1420-4584-a85b-768649d3252e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:27.828301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-dec17a8a-1420-4584-a85b-768649d3252e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045722 Sep 08 14:02:27.828866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 720/1335] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:27 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:27.948574 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-70b4dfe3-7d6d-46a6-be63-b4dc84965dac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:27.950789 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-70b4dfe3-7d6d-46a6-be63-b4dc84965dac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:27.952680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:27.957451 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-70b4dfe3-7d6d-46a6-be63-b4dc84965dac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:27.957929 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-70b4dfe3-7d6d-46a6-be63-b4dc84965dac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:27.960496 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-70b4dfe3-7d6d-46a6-be63-b4dc84965dac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:27.960496 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-70b4dfe3-7d6d-46a6-be63-b4dc84965dac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:27.960496 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-70b4dfe3-7d6d-46a6-be63-b4dc84965dac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:27.960496 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-70b4dfe3-7d6d-46a6-be63-b4dc84965dac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:27.991428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-70b4dfe3-7d6d-46a6-be63-b4dc84965dac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046799 Sep 08 14:02:27.991992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 616/1336] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:27 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:28.422691 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-aa2be351-d2de-4807-ae55-01e9d43bbe3c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:28.423438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-aa2be351-d2de-4807-ae55-01e9d43bbe3c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:28.430065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa2be351-d2de-4807-ae55-01e9d43bbe3c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:28.430581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa2be351-d2de-4807-ae55-01e9d43bbe3c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:28.431013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa2be351-d2de-4807-ae55-01e9d43bbe3c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:28.431693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa2be351-d2de-4807-ae55-01e9d43bbe3c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:28.432144 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa2be351-d2de-4807-ae55-01e9d43bbe3c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:28.432692 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa2be351-d2de-4807-ae55-01e9d43bbe3c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:28.488051 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-68d7ff94-fff2-4abd-ba92-c31310b05182 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:28.490231 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-68d7ff94-fff2-4abd-ba92-c31310b05182 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:28.497557 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-68d7ff94-fff2-4abd-ba92-c31310b05182 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:28.497708 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-aa2be351-d2de-4807-ae55-01e9d43bbe3c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:28.498039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-68d7ff94-fff2-4abd-ba92-c31310b05182 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:28.498514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-68d7ff94-fff2-4abd-ba92-c31310b05182 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:28.499634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-68d7ff94-fff2-4abd-ba92-c31310b05182 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:28.499875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-68d7ff94-fff2-4abd-ba92-c31310b05182 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:28.500297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-68d7ff94-fff2-4abd-ba92-c31310b05182 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:28.506501 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-aa2be351-d2de-4807-ae55-01e9d43bbe3c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.089457 Sep 08 14:02:28.507679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 721/1337] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:28 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:28.530236 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-68d7ff94-fff2-4abd-ba92-c31310b05182 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046059 Sep 08 14:02:28.530784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 617/1338] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:28 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:28.668776 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0d344328-0743-407a-b4c2-c13d503626f8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:28.669431 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0d344328-0743-407a-b4c2-c13d503626f8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:28.675943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d344328-0743-407a-b4c2-c13d503626f8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:28.676372 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d344328-0743-407a-b4c2-c13d503626f8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:28.676910 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d344328-0743-407a-b4c2-c13d503626f8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:28.677667 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d344328-0743-407a-b4c2-c13d503626f8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:28.678159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d344328-0743-407a-b4c2-c13d503626f8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:28.679098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d344328-0743-407a-b4c2-c13d503626f8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:28.808945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0d344328-0743-407a-b4c2-c13d503626f8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:28.816630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0d344328-0743-407a-b4c2-c13d503626f8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.152942 Sep 08 14:02:28.817199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 722/1339] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:28 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:28.839998 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7f2023d8-d82e-4e7f-b182-88458a4bd882 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:28.842296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7f2023d8-d82e-4e7f-b182-88458a4bd882 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:28.848124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7f2023d8-d82e-4e7f-b182-88458a4bd882 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:28.848596 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7f2023d8-d82e-4e7f-b182-88458a4bd882 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:28.849022 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7f2023d8-d82e-4e7f-b182-88458a4bd882 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:28.849691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7f2023d8-d82e-4e7f-b182-88458a4bd882 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:28.850156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7f2023d8-d82e-4e7f-b182-88458a4bd882 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:28.851148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7f2023d8-d82e-4e7f-b182-88458a4bd882 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:28.880277 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7f2023d8-d82e-4e7f-b182-88458a4bd882 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044286 Sep 08 14:02:28.880824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 618/1340] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:28 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:29.005708 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ea15de2c-beea-4ba6-a486-3719e8c7dd19 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:29.007846 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ea15de2c-beea-4ba6-a486-3719e8c7dd19 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:29.014360 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ea15de2c-beea-4ba6-a486-3719e8c7dd19 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:29.014858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ea15de2c-beea-4ba6-a486-3719e8c7dd19 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:29.015295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ea15de2c-beea-4ba6-a486-3719e8c7dd19 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:29.016008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ea15de2c-beea-4ba6-a486-3719e8c7dd19 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:29.016454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ea15de2c-beea-4ba6-a486-3719e8c7dd19 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:29.016880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ea15de2c-beea-4ba6-a486-3719e8c7dd19 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:29.049122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ea15de2c-beea-4ba6-a486-3719e8c7dd19 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048616 Sep 08 14:02:29.049710 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 723/1341] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:28 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:29.529712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c2dee3de-482c-437c-b04f-da4d3ceb619b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:29.530650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c2dee3de-482c-437c-b04f-da4d3ceb619b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:29.537239 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2dee3de-482c-437c-b04f-da4d3ceb619b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:29.537823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2dee3de-482c-437c-b04f-da4d3ceb619b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:29.539013 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2dee3de-482c-437c-b04f-da4d3ceb619b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:29.539743 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2dee3de-482c-437c-b04f-da4d3ceb619b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:29.540203 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2dee3de-482c-437c-b04f-da4d3ceb619b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:29.540631 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2dee3de-482c-437c-b04f-da4d3ceb619b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:29.546277 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-de10d9a5-cd35-4d7f-b906-9a38e0ef4b6d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:29.548926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-de10d9a5-cd35-4d7f-b906-9a38e0ef4b6d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:29.556168 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de10d9a5-cd35-4d7f-b906-9a38e0ef4b6d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:29.556697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de10d9a5-cd35-4d7f-b906-9a38e0ef4b6d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:29.557144 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de10d9a5-cd35-4d7f-b906-9a38e0ef4b6d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:29.557857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de10d9a5-cd35-4d7f-b906-9a38e0ef4b6d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:29.558351 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de10d9a5-cd35-4d7f-b906-9a38e0ef4b6d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:29.558777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de10d9a5-cd35-4d7f-b906-9a38e0ef4b6d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:29.597595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-de10d9a5-cd35-4d7f-b906-9a38e0ef4b6d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057532 Sep 08 14:02:29.598552 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 724/1342] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:29 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:29.614202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c2dee3de-482c-437c-b04f-da4d3ceb619b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:29.622425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c2dee3de-482c-437c-b04f-da4d3ceb619b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.096662 Sep 08 14:02:29.623015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 619/1343] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:29 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:29.840903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f5fb7575-0d08-4c52-b48c-84abcd8045c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:29.841551 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f5fb7575-0d08-4c52-b48c-84abcd8045c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:29.847642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f5fb7575-0d08-4c52-b48c-84abcd8045c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:29.848134 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f5fb7575-0d08-4c52-b48c-84abcd8045c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:29.848689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f5fb7575-0d08-4c52-b48c-84abcd8045c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:29.849259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f5fb7575-0d08-4c52-b48c-84abcd8045c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:29.849715 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f5fb7575-0d08-4c52-b48c-84abcd8045c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:29.850175 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f5fb7575-0d08-4c52-b48c-84abcd8045c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:29.893099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bc1bb6a7-d42c-4c99-bc30-cfc7b7c9cff4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:29.895474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bc1bb6a7-d42c-4c99-bc30-cfc7b7c9cff4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:29.903020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc1bb6a7-d42c-4c99-bc30-cfc7b7c9cff4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:29.903513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc1bb6a7-d42c-4c99-bc30-cfc7b7c9cff4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:29.903950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc1bb6a7-d42c-4c99-bc30-cfc7b7c9cff4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:29.904656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc1bb6a7-d42c-4c99-bc30-cfc7b7c9cff4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:29.905108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc1bb6a7-d42c-4c99-bc30-cfc7b7c9cff4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:29.905529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc1bb6a7-d42c-4c99-bc30-cfc7b7c9cff4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:29.942481 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bc1bb6a7-d42c-4c99-bc30-cfc7b7c9cff4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053498 Sep 08 14:02:29.943078 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 620/1344] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:29 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:29.989946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f5fb7575-0d08-4c52-b48c-84abcd8045c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:30.008111 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f5fb7575-0d08-4c52-b48c-84abcd8045c8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.170975 Sep 08 14:02:30.008703 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 725/1345] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:29 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:30.065042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-25bee09e-f099-4582-a5b1-dd0cad6f3d0d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:30.068711 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-25bee09e-f099-4582-a5b1-dd0cad6f3d0d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:30.078020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-25bee09e-f099-4582-a5b1-dd0cad6f3d0d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:30.078547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-25bee09e-f099-4582-a5b1-dd0cad6f3d0d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:30.078979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-25bee09e-f099-4582-a5b1-dd0cad6f3d0d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:30.079678 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-25bee09e-f099-4582-a5b1-dd0cad6f3d0d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:30.080126 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-25bee09e-f099-4582-a5b1-dd0cad6f3d0d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:30.080542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-25bee09e-f099-4582-a5b1-dd0cad6f3d0d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:30.114417 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-25bee09e-f099-4582-a5b1-dd0cad6f3d0d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055496 Sep 08 14:02:30.114920 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 621/1346] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:30 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:30.611928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-997ebde6-a152-434a-9932-5402a98fffc4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:30.614204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-997ebde6-a152-434a-9932-5402a98fffc4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:30.623151 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-997ebde6-a152-434a-9932-5402a98fffc4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:30.623699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-997ebde6-a152-434a-9932-5402a98fffc4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:30.624176 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-997ebde6-a152-434a-9932-5402a98fffc4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:30.624887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-997ebde6-a152-434a-9932-5402a98fffc4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:30.625402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-997ebde6-a152-434a-9932-5402a98fffc4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:30.625834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-997ebde6-a152-434a-9932-5402a98fffc4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:30.628806 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:30.647200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1c12c407-fb26-4cbb-a3a4-9a694bb2b286 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:30.647929 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1c12c407-fb26-4cbb-a3a4-9a694bb2b286 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:30.654608 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c12c407-fb26-4cbb-a3a4-9a694bb2b286 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:30.655095 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c12c407-fb26-4cbb-a3a4-9a694bb2b286 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:30.655540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c12c407-fb26-4cbb-a3a4-9a694bb2b286 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:30.656253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c12c407-fb26-4cbb-a3a4-9a694bb2b286 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:30.656760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c12c407-fb26-4cbb-a3a4-9a694bb2b286 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:30.657136 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c12c407-fb26-4cbb-a3a4-9a694bb2b286 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:30.660851 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-997ebde6-a152-434a-9932-5402a98fffc4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052624 Sep 08 14:02:30.661711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 726/1347] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:30 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:30.661850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:30.876202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1c12c407-fb26-4cbb-a3a4-9a694bb2b286 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:30.884428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1c12c407-fb26-4cbb-a3a4-9a694bb2b286 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.241128 Sep 08 14:02:30.885001 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 622/1348] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:30 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:30.955770 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-50142070-0278-43c1-bae3-33ac9340d2d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:30.957896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-50142070-0278-43c1-bae3-33ac9340d2d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:30.965039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50142070-0278-43c1-bae3-33ac9340d2d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:30.965518 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50142070-0278-43c1-bae3-33ac9340d2d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:30.965949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50142070-0278-43c1-bae3-33ac9340d2d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:30.966666 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50142070-0278-43c1-bae3-33ac9340d2d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:30.967112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50142070-0278-43c1-bae3-33ac9340d2d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:30.967531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50142070-0278-43c1-bae3-33ac9340d2d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:30.997835 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-50142070-0278-43c1-bae3-33ac9340d2d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045951 Sep 08 14:02:30.998437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 727/1349] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:30 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:31.035387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9ab7e204-6a74-4185-badc-756fae10b56c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:31.035992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9ab7e204-6a74-4185-badc-756fae10b56c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:31.045626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9ab7e204-6a74-4185-badc-756fae10b56c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:31.046121 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9ab7e204-6a74-4185-badc-756fae10b56c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:31.046627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9ab7e204-6a74-4185-badc-756fae10b56c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:31.047536 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9ab7e204-6a74-4185-badc-756fae10b56c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:31.048020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9ab7e204-6a74-4185-badc-756fae10b56c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:31.048450 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9ab7e204-6a74-4185-badc-756fae10b56c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:31.128043 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-edeb4f76-da83-4982-bfaa-b3a997886f1e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:31.130094 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-edeb4f76-da83-4982-bfaa-b3a997886f1e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:31.137748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-edeb4f76-da83-4982-bfaa-b3a997886f1e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:31.138360 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-edeb4f76-da83-4982-bfaa-b3a997886f1e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:31.138699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-edeb4f76-da83-4982-bfaa-b3a997886f1e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:31.139438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-edeb4f76-da83-4982-bfaa-b3a997886f1e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:31.139929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-edeb4f76-da83-4982-bfaa-b3a997886f1e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:31.140292 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-edeb4f76-da83-4982-bfaa-b3a997886f1e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:31.174674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-edeb4f76-da83-4982-bfaa-b3a997886f1e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050933 Sep 08 14:02:31.175261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 728/1350] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:31 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:02:31.200489 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9ab7e204-6a74-4185-badc-756fae10b56c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:31.208514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9ab7e204-6a74-4185-badc-756fae10b56c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.177022 Sep 08 14:02:31.209082 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 623/1351] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:31 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:31.677750 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1979c158-bd47-4ae8-9917-23157096dc7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:31.680388 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1979c158-bd47-4ae8-9917-23157096dc7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:31.687457 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1979c158-bd47-4ae8-9917-23157096dc7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:31.687954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1979c158-bd47-4ae8-9917-23157096dc7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:31.690042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1979c158-bd47-4ae8-9917-23157096dc7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:31.690042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1979c158-bd47-4ae8-9917-23157096dc7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:31.690042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1979c158-bd47-4ae8-9917-23157096dc7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:31.690042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1979c158-bd47-4ae8-9917-23157096dc7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:31.729359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1979c158-bd47-4ae8-9917-23157096dc7a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056132 Sep 08 14:02:31.730193 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 729/1352] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:31 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:31.908262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1b3b3751-0755-4aed-b3fd-2cb00d2fdfd4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:31.908861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1b3b3751-0755-4aed-b3fd-2cb00d2fdfd4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:31.915981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b3b3751-0755-4aed-b3fd-2cb00d2fdfd4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:31.920104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b3b3751-0755-4aed-b3fd-2cb00d2fdfd4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:31.920104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b3b3751-0755-4aed-b3fd-2cb00d2fdfd4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:31.920104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b3b3751-0755-4aed-b3fd-2cb00d2fdfd4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:31.920104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b3b3751-0755-4aed-b3fd-2cb00d2fdfd4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:31.920646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b3b3751-0755-4aed-b3fd-2cb00d2fdfd4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:32.001905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1b3b3751-0755-4aed-b3fd-2cb00d2fdfd4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:32.011150 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1b3b3751-0755-4aed-b3fd-2cb00d2fdfd4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.106555 Sep 08 14:02:32.012043 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 624/1353] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:31 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:32.012245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-de4c8377-25ac-4668-b2fa-862e1a718d8f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:32.015246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-de4c8377-25ac-4668-b2fa-862e1a718d8f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:32.021620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de4c8377-25ac-4668-b2fa-862e1a718d8f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:32.021907 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de4c8377-25ac-4668-b2fa-862e1a718d8f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:32.022314 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de4c8377-25ac-4668-b2fa-862e1a718d8f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:32.023175 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de4c8377-25ac-4668-b2fa-862e1a718d8f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:32.023538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de4c8377-25ac-4668-b2fa-862e1a718d8f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:32.023966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de4c8377-25ac-4668-b2fa-862e1a718d8f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:32.064188 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-de4c8377-25ac-4668-b2fa-862e1a718d8f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056822 Sep 08 14:02:32.064761 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 730/1354] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:32 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:32.187641 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-50fe1bcb-ae33-4c95-9fb1-73389fd64f8c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:32.189727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-50fe1bcb-ae33-4c95-9fb1-73389fd64f8c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:32.205843 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-50fe1bcb-ae33-4c95-9fb1-73389fd64f8c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:32.207268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-50fe1bcb-ae33-4c95-9fb1-73389fd64f8c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:32.207709 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-50fe1bcb-ae33-4c95-9fb1-73389fd64f8c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:32.208424 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-50fe1bcb-ae33-4c95-9fb1-73389fd64f8c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:32.208877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-50fe1bcb-ae33-4c95-9fb1-73389fd64f8c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:32.209441 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-50fe1bcb-ae33-4c95-9fb1-73389fd64f8c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:32.233614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ddebde3c-5cb2-479f-834f-de2b242faffb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:32.234402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ddebde3c-5cb2-479f-834f-de2b242faffb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:32.242101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ddebde3c-5cb2-479f-834f-de2b242faffb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:32.242617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ddebde3c-5cb2-479f-834f-de2b242faffb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:32.243057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ddebde3c-5cb2-479f-834f-de2b242faffb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:32.243762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ddebde3c-5cb2-479f-834f-de2b242faffb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:32.244212 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ddebde3c-5cb2-479f-834f-de2b242faffb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:32.244633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ddebde3c-5cb2-479f-834f-de2b242faffb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:32.246421 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-50fe1bcb-ae33-4c95-9fb1-73389fd64f8c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.062880 Sep 08 14:02:32.247450 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 625/1355] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:32 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:32.390072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ddebde3c-5cb2-479f-834f-de2b242faffb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:32.398783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ddebde3c-5cb2-479f-834f-de2b242faffb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.169185 Sep 08 14:02:32.399359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 731/1356] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:32 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:32.744073 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8c07383d-8df6-4a9a-b45d-0b7c2670bf65 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:32.746242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8c07383d-8df6-4a9a-b45d-0b7c2670bf65 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:32.752592 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8c07383d-8df6-4a9a-b45d-0b7c2670bf65 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:32.753072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8c07383d-8df6-4a9a-b45d-0b7c2670bf65 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:32.753509 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8c07383d-8df6-4a9a-b45d-0b7c2670bf65 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:32.754228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8c07383d-8df6-4a9a-b45d-0b7c2670bf65 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:32.754679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8c07383d-8df6-4a9a-b45d-0b7c2670bf65 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:32.755101 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8c07383d-8df6-4a9a-b45d-0b7c2670bf65 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:32.785403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8c07383d-8df6-4a9a-b45d-0b7c2670bf65 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046282 Sep 08 14:02:32.785979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 626/1357] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:32 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:33.038752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-db206711-384b-406b-9bfa-18dfd670f97b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:33.039368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-db206711-384b-406b-9bfa-18dfd670f97b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:33.046019 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db206711-384b-406b-9bfa-18dfd670f97b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:33.046544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db206711-384b-406b-9bfa-18dfd670f97b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:33.046999 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db206711-384b-406b-9bfa-18dfd670f97b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:33.047693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db206711-384b-406b-9bfa-18dfd670f97b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:33.048149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db206711-384b-406b-9bfa-18dfd670f97b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:33.048608 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db206711-384b-406b-9bfa-18dfd670f97b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:33.080336 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-178a8e6d-d466-4331-8567-aaa17b9cdfe4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:33.083571 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-178a8e6d-d466-4331-8567-aaa17b9cdfe4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:33.095460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-178a8e6d-d466-4331-8567-aaa17b9cdfe4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:33.096303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-178a8e6d-d466-4331-8567-aaa17b9cdfe4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:33.097053 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-178a8e6d-d466-4331-8567-aaa17b9cdfe4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:33.098327 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-178a8e6d-d466-4331-8567-aaa17b9cdfe4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:33.099150 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-178a8e6d-d466-4331-8567-aaa17b9cdfe4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:33.099991 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-178a8e6d-d466-4331-8567-aaa17b9cdfe4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:33.119643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-db206711-384b-406b-9bfa-18dfd670f97b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:33.134944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-db206711-384b-406b-9bfa-18dfd670f97b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.100772 Sep 08 14:02:33.135916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 732/1358] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:33 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:33.139061 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-178a8e6d-d466-4331-8567-aaa17b9cdfe4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064669 Sep 08 14:02:33.144193 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 627/1359] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:33 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:33.261919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e5e76f54-4327-4993-b7d6-f0632664b12a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:33.264105 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e5e76f54-4327-4993-b7d6-f0632664b12a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:33.270166 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e5e76f54-4327-4993-b7d6-f0632664b12a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:33.276291 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e5e76f54-4327-4993-b7d6-f0632664b12a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:33.276291 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e5e76f54-4327-4993-b7d6-f0632664b12a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:33.276291 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e5e76f54-4327-4993-b7d6-f0632664b12a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:33.276291 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e5e76f54-4327-4993-b7d6-f0632664b12a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:33.276693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e5e76f54-4327-4993-b7d6-f0632664b12a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:33.307640 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e5e76f54-4327-4993-b7d6-f0632664b12a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050442 Sep 08 14:02:33.308252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 733/1360] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:33 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:33.423924 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1ea59bd5-14b8-442d-8279-a38cfbe6a7a4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:33.424535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1ea59bd5-14b8-442d-8279-a38cfbe6a7a4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:33.431110 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1ea59bd5-14b8-442d-8279-a38cfbe6a7a4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:33.431728 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1ea59bd5-14b8-442d-8279-a38cfbe6a7a4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:33.432168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1ea59bd5-14b8-442d-8279-a38cfbe6a7a4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:33.432877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1ea59bd5-14b8-442d-8279-a38cfbe6a7a4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:33.433332 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1ea59bd5-14b8-442d-8279-a38cfbe6a7a4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:33.433785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1ea59bd5-14b8-442d-8279-a38cfbe6a7a4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:33.582975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1ea59bd5-14b8-442d-8279-a38cfbe6a7a4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:33.586698 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1ea59bd5-14b8-442d-8279-a38cfbe6a7a4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.166378 Sep 08 14:02:33.587294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 628/1361] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:33 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:33.799195 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-42621c6c-ce32-4139-bd55-7d711b72f54f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:33.801376 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-42621c6c-ce32-4139-bd55-7d711b72f54f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:33.808128 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-42621c6c-ce32-4139-bd55-7d711b72f54f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:33.808619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-42621c6c-ce32-4139-bd55-7d711b72f54f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:33.809058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-42621c6c-ce32-4139-bd55-7d711b72f54f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:33.809877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-42621c6c-ce32-4139-bd55-7d711b72f54f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:33.810360 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-42621c6c-ce32-4139-bd55-7d711b72f54f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:33.810788 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-42621c6c-ce32-4139-bd55-7d711b72f54f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:33.848469 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-42621c6c-ce32-4139-bd55-7d711b72f54f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053410 Sep 08 14:02:33.849062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 734/1362] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:33 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:34.153056 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4f94f2da-9cec-4f6b-b285-a261f88a34ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:34.155132 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4f94f2da-9cec-4f6b-b285-a261f88a34ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:34.160927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-69b995e7-91c5-416a-a287-ce09912cf741 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:34.161558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-69b995e7-91c5-416a-a287-ce09912cf741 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:34.161823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f94f2da-9cec-4f6b-b285-a261f88a34ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:34.162870 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f94f2da-9cec-4f6b-b285-a261f88a34ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:34.163303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f94f2da-9cec-4f6b-b285-a261f88a34ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:34.164005 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f94f2da-9cec-4f6b-b285-a261f88a34ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:34.164454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f94f2da-9cec-4f6b-b285-a261f88a34ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:34.164873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f94f2da-9cec-4f6b-b285-a261f88a34ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:34.169036 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69b995e7-91c5-416a-a287-ce09912cf741 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:34.169532 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69b995e7-91c5-416a-a287-ce09912cf741 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:34.169982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69b995e7-91c5-416a-a287-ce09912cf741 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:34.170729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69b995e7-91c5-416a-a287-ce09912cf741 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:34.171189 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69b995e7-91c5-416a-a287-ce09912cf741 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:34.171619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69b995e7-91c5-416a-a287-ce09912cf741 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:34.196541 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4f94f2da-9cec-4f6b-b285-a261f88a34ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047648 Sep 08 14:02:34.197126 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 629/1363] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:34 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:34.246484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-69b995e7-91c5-416a-a287-ce09912cf741 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:34.254535 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-69b995e7-91c5-416a-a287-ce09912cf741 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.097590 Sep 08 14:02:34.255102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 735/1364] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:34 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:34.320537 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1c4d8205-f5ce-4563-b095-4da951e90d8e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:34.322685 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1c4d8205-f5ce-4563-b095-4da951e90d8e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:34.328603 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c4d8205-f5ce-4563-b095-4da951e90d8e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:34.329086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c4d8205-f5ce-4563-b095-4da951e90d8e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:34.329521 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c4d8205-f5ce-4563-b095-4da951e90d8e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:34.330239 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c4d8205-f5ce-4563-b095-4da951e90d8e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:34.330690 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c4d8205-f5ce-4563-b095-4da951e90d8e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:34.331113 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c4d8205-f5ce-4563-b095-4da951e90d8e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:34.361334 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1c4d8205-f5ce-4563-b095-4da951e90d8e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044744 Sep 08 14:02:34.361893 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 630/1365] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:34 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:34.611719 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d477e729-f69e-4cdb-98c4-5262c84dd113 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:34.612855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d477e729-f69e-4cdb-98c4-5262c84dd113 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:34.624119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d477e729-f69e-4cdb-98c4-5262c84dd113 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:34.624993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d477e729-f69e-4cdb-98c4-5262c84dd113 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:34.625475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d477e729-f69e-4cdb-98c4-5262c84dd113 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:34.627050 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d477e729-f69e-4cdb-98c4-5262c84dd113 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:34.627904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d477e729-f69e-4cdb-98c4-5262c84dd113 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:34.628710 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d477e729-f69e-4cdb-98c4-5262c84dd113 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:34.778976 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d477e729-f69e-4cdb-98c4-5262c84dd113 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:34.787996 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d477e729-f69e-4cdb-98c4-5262c84dd113 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.181328 Sep 08 14:02:34.788581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 736/1366] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:34 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:34.864967 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0152a851-19ad-4d01-9743-e4c4ec09da34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:34.867267 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0152a851-19ad-4d01-9743-e4c4ec09da34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:34.873752 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0152a851-19ad-4d01-9743-e4c4ec09da34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:34.874265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0152a851-19ad-4d01-9743-e4c4ec09da34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:34.874699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0152a851-19ad-4d01-9743-e4c4ec09da34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:34.875399 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0152a851-19ad-4d01-9743-e4c4ec09da34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:34.875847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0152a851-19ad-4d01-9743-e4c4ec09da34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:34.876263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0152a851-19ad-4d01-9743-e4c4ec09da34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:34.908942 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0152a851-19ad-4d01-9743-e4c4ec09da34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049295 Sep 08 14:02:34.909557 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 631/1367] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:34 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:35.212027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-01b085b1-7582-40bc-8af7-739986435dc3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:35.214184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-01b085b1-7582-40bc-8af7-739986435dc3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:35.220712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01b085b1-7582-40bc-8af7-739986435dc3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:35.221208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01b085b1-7582-40bc-8af7-739986435dc3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:35.221650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01b085b1-7582-40bc-8af7-739986435dc3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:35.222381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01b085b1-7582-40bc-8af7-739986435dc3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:35.222842 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01b085b1-7582-40bc-8af7-739986435dc3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:35.223275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01b085b1-7582-40bc-8af7-739986435dc3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:35.254412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-01b085b1-7582-40bc-8af7-739986435dc3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047508 Sep 08 14:02:35.254988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 737/1368] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:35 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:35.277723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-79cac578-b694-49e9-964e-addb972c069a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:35.278425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-79cac578-b694-49e9-964e-addb972c069a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:35.284937 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79cac578-b694-49e9-964e-addb972c069a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:35.285432 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79cac578-b694-49e9-964e-addb972c069a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:35.285860 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79cac578-b694-49e9-964e-addb972c069a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:35.286772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79cac578-b694-49e9-964e-addb972c069a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:35.288765 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79cac578-b694-49e9-964e-addb972c069a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:35.288765 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-79cac578-b694-49e9-964e-addb972c069a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:35.351764 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-79cac578-b694-49e9-964e-addb972c069a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:35.359444 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-79cac578-b694-49e9-964e-addb972c069a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.085958 Sep 08 14:02:35.360000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 632/1369] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:35 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:35.380713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bb0b2e14-ccf1-41d6-a8db-465c71fe0a4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:35.383697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bb0b2e14-ccf1-41d6-a8db-465c71fe0a4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:35.389933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb0b2e14-ccf1-41d6-a8db-465c71fe0a4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:35.390461 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb0b2e14-ccf1-41d6-a8db-465c71fe0a4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:35.390906 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb0b2e14-ccf1-41d6-a8db-465c71fe0a4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:35.391610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb0b2e14-ccf1-41d6-a8db-465c71fe0a4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:35.392068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb0b2e14-ccf1-41d6-a8db-465c71fe0a4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:35.392497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb0b2e14-ccf1-41d6-a8db-465c71fe0a4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:35.423020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bb0b2e14-ccf1-41d6-a8db-465c71fe0a4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051537 Sep 08 14:02:35.423578 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 738/1370] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:35 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:02:35.811248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c74bbcf8-83f9-4cb4-be01-3c11137ca8a5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:35.811840 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c74bbcf8-83f9-4cb4-be01-3c11137ca8a5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:35.817833 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c74bbcf8-83f9-4cb4-be01-3c11137ca8a5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:35.818349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c74bbcf8-83f9-4cb4-be01-3c11137ca8a5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:35.818785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c74bbcf8-83f9-4cb4-be01-3c11137ca8a5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:35.819464 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c74bbcf8-83f9-4cb4-be01-3c11137ca8a5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:35.819910 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c74bbcf8-83f9-4cb4-be01-3c11137ca8a5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:35.820318 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c74bbcf8-83f9-4cb4-be01-3c11137ca8a5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:35.923485 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-90c985f5-46e1-4ecd-894e-6b277ef82fc9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:35.925619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-90c985f5-46e1-4ecd-894e-6b277ef82fc9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:35.932981 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-90c985f5-46e1-4ecd-894e-6b277ef82fc9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:35.933504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-90c985f5-46e1-4ecd-894e-6b277ef82fc9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:35.933948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-90c985f5-46e1-4ecd-894e-6b277ef82fc9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:35.934696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-90c985f5-46e1-4ecd-894e-6b277ef82fc9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:35.935159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-90c985f5-46e1-4ecd-894e-6b277ef82fc9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:35.935589 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-90c985f5-46e1-4ecd-894e-6b277ef82fc9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:35.968139 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-90c985f5-46e1-4ecd-894e-6b277ef82fc9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048522 Sep 08 14:02:35.968708 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 739/1371] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:35 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:35.974253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c74bbcf8-83f9-4cb4-be01-3c11137ca8a5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:35.983079 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c74bbcf8-83f9-4cb4-be01-3c11137ca8a5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.175709 Sep 08 14:02:35.983654 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 633/1372] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:35 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:36.267739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2e007388-dc47-4296-948f-b4ed20778690 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:36.269781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2e007388-dc47-4296-948f-b4ed20778690 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:36.272593 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:36.280300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e007388-dc47-4296-948f-b4ed20778690 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:36.282104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e007388-dc47-4296-948f-b4ed20778690 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:36.282104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e007388-dc47-4296-948f-b4ed20778690 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:36.282104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e007388-dc47-4296-948f-b4ed20778690 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:36.282438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e007388-dc47-4296-948f-b4ed20778690 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:36.282854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e007388-dc47-4296-948f-b4ed20778690 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:36.314914 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2e007388-dc47-4296-948f-b4ed20778690 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051192 Sep 08 14:02:36.315476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 740/1373] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:36 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:36.383311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1cd5dec6-ad72-4408-ba03-1a117135c9de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:36.383892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1cd5dec6-ad72-4408-ba03-1a117135c9de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:36.390073 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1cd5dec6-ad72-4408-ba03-1a117135c9de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:36.390579 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1cd5dec6-ad72-4408-ba03-1a117135c9de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:36.391009 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1cd5dec6-ad72-4408-ba03-1a117135c9de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:36.391691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1cd5dec6-ad72-4408-ba03-1a117135c9de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:36.392131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1cd5dec6-ad72-4408-ba03-1a117135c9de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:36.392545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1cd5dec6-ad72-4408-ba03-1a117135c9de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:36.437299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fb0d2ff1-2da8-4681-b557-eb3d918b43ff tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:36.439644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fb0d2ff1-2da8-4681-b557-eb3d918b43ff tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:36.448344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb0d2ff1-2da8-4681-b557-eb3d918b43ff tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:36.452137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb0d2ff1-2da8-4681-b557-eb3d918b43ff tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:36.452137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb0d2ff1-2da8-4681-b557-eb3d918b43ff tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:36.452137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb0d2ff1-2da8-4681-b557-eb3d918b43ff tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:36.452137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb0d2ff1-2da8-4681-b557-eb3d918b43ff tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:36.452616 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb0d2ff1-2da8-4681-b557-eb3d918b43ff tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:36.457480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1cd5dec6-ad72-4408-ba03-1a117135c9de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:36.467710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1cd5dec6-ad72-4408-ba03-1a117135c9de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.089056 Sep 08 14:02:36.468411 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 634/1374] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:36 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:36.486119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fb0d2ff1-2da8-4681-b557-eb3d918b43ff tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052833 Sep 08 14:02:36.486706 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 741/1375] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:36 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:02:36.981934 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bb31f480-97d8-46bd-b535-a75d710a73df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:36.984043 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bb31f480-97d8-46bd-b535-a75d710a73df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:36.990530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bb31f480-97d8-46bd-b535-a75d710a73df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:36.991005 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bb31f480-97d8-46bd-b535-a75d710a73df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:36.991435 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bb31f480-97d8-46bd-b535-a75d710a73df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:36.992117 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bb31f480-97d8-46bd-b535-a75d710a73df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:36.992557 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bb31f480-97d8-46bd-b535-a75d710a73df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:36.992973 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bb31f480-97d8-46bd-b535-a75d710a73df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:37.007734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d3c1e37d-43a3-494a-a9b3-664a57bd2d65 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:37.008347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d3c1e37d-43a3-494a-a9b3-664a57bd2d65 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:37.014373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d3c1e37d-43a3-494a-a9b3-664a57bd2d65 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:37.014866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d3c1e37d-43a3-494a-a9b3-664a57bd2d65 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:37.015342 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d3c1e37d-43a3-494a-a9b3-664a57bd2d65 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:37.016043 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d3c1e37d-43a3-494a-a9b3-664a57bd2d65 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:37.016507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d3c1e37d-43a3-494a-a9b3-664a57bd2d65 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:37.016935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d3c1e37d-43a3-494a-a9b3-664a57bd2d65 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:37.023767 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bb31f480-97d8-46bd-b535-a75d710a73df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045888 Sep 08 14:02:37.024344 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 635/1376] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:36 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:37.152358 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d3c1e37d-43a3-494a-a9b3-664a57bd2d65 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:37.154564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:37.160777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d3c1e37d-43a3-494a-a9b3-664a57bd2d65 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.156919 Sep 08 14:02:37.161352 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 742/1377] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:37 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:37.327653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-aa6217aa-1eda-4664-b783-b71f4046d2f7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:37.329850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-aa6217aa-1eda-4664-b783-b71f4046d2f7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:37.336604 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa6217aa-1eda-4664-b783-b71f4046d2f7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:37.337100 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa6217aa-1eda-4664-b783-b71f4046d2f7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:37.337701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa6217aa-1eda-4664-b783-b71f4046d2f7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:37.343154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa6217aa-1eda-4664-b783-b71f4046d2f7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:37.343154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa6217aa-1eda-4664-b783-b71f4046d2f7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:37.343154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa6217aa-1eda-4664-b783-b71f4046d2f7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:37.371443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-aa6217aa-1eda-4664-b783-b71f4046d2f7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047763 Sep 08 14:02:37.372030 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 636/1378] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:37 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:37.491673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2a89888e-5f19-4d09-8efd-b0ab9ac65c1f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:37.492445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2a89888e-5f19-4d09-8efd-b0ab9ac65c1f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:37.500111 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d94151d6-dab7-493f-bda1-bf5bee268312 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:37.500526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a89888e-5f19-4d09-8efd-b0ab9ac65c1f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:37.500648 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a89888e-5f19-4d09-8efd-b0ab9ac65c1f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:37.501056 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a89888e-5f19-4d09-8efd-b0ab9ac65c1f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:37.502022 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a89888e-5f19-4d09-8efd-b0ab9ac65c1f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:37.502575 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a89888e-5f19-4d09-8efd-b0ab9ac65c1f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:37.502891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d94151d6-dab7-493f-bda1-bf5bee268312 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:37.503087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a89888e-5f19-4d09-8efd-b0ab9ac65c1f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:37.509937 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d94151d6-dab7-493f-bda1-bf5bee268312 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:37.510459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d94151d6-dab7-493f-bda1-bf5bee268312 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:37.510922 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d94151d6-dab7-493f-bda1-bf5bee268312 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:37.511677 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d94151d6-dab7-493f-bda1-bf5bee268312 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:37.512100 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d94151d6-dab7-493f-bda1-bf5bee268312 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:37.512514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d94151d6-dab7-493f-bda1-bf5bee268312 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:37.546827 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d94151d6-dab7-493f-bda1-bf5bee268312 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050399 Sep 08 14:02:37.547810 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 637/1379] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:37 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:37.572279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2a89888e-5f19-4d09-8efd-b0ab9ac65c1f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:37.580251 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2a89888e-5f19-4d09-8efd-b0ab9ac65c1f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.093156 Sep 08 14:02:37.580826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 743/1380] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:37 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:38.037951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c348a477-33c3-4508-aabf-d3214c9b04c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:38.040143 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c348a477-33c3-4508-aabf-d3214c9b04c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:38.047260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c348a477-33c3-4508-aabf-d3214c9b04c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:38.047743 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c348a477-33c3-4508-aabf-d3214c9b04c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:38.048201 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c348a477-33c3-4508-aabf-d3214c9b04c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:38.048896 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c348a477-33c3-4508-aabf-d3214c9b04c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:38.049346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c348a477-33c3-4508-aabf-d3214c9b04c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:38.049760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c348a477-33c3-4508-aabf-d3214c9b04c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:38.076458 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:38.081334 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c348a477-33c3-4508-aabf-d3214c9b04c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048208 Sep 08 14:02:38.081902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 638/1381] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:38 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:38.189690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8c81f18b-d1fe-4f50-bb85-707150b61c73 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:38.190238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8c81f18b-d1fe-4f50-bb85-707150b61c73 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:38.197410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c81f18b-d1fe-4f50-bb85-707150b61c73 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:38.197912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c81f18b-d1fe-4f50-bb85-707150b61c73 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:38.198384 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c81f18b-d1fe-4f50-bb85-707150b61c73 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:38.199101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c81f18b-d1fe-4f50-bb85-707150b61c73 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:38.199570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c81f18b-d1fe-4f50-bb85-707150b61c73 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:38.200343 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c81f18b-d1fe-4f50-bb85-707150b61c73 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:38.341202 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8c81f18b-d1fe-4f50-bb85-707150b61c73 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:38.350004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8c81f18b-d1fe-4f50-bb85-707150b61c73 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.163648 Sep 08 14:02:38.350610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 744/1382] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:38 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:38.386536 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6445b586-9b91-44db-95d9-1a1e72bc613c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:38.388657 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6445b586-9b91-44db-95d9-1a1e72bc613c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:38.395196 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6445b586-9b91-44db-95d9-1a1e72bc613c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:38.395671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6445b586-9b91-44db-95d9-1a1e72bc613c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:38.396670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6445b586-9b91-44db-95d9-1a1e72bc613c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:38.397353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6445b586-9b91-44db-95d9-1a1e72bc613c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:38.397807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6445b586-9b91-44db-95d9-1a1e72bc613c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:38.398244 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6445b586-9b91-44db-95d9-1a1e72bc613c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:38.429418 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6445b586-9b91-44db-95d9-1a1e72bc613c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049834 Sep 08 14:02:38.430026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 639/1383] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:38 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:38.560429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9be5065e-b602-434c-ba73-3813499010b7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:38.562629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9be5065e-b602-434c-ba73-3813499010b7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:38.569418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9be5065e-b602-434c-ba73-3813499010b7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:38.569907 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9be5065e-b602-434c-ba73-3813499010b7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:38.570378 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9be5065e-b602-434c-ba73-3813499010b7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:38.571097 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9be5065e-b602-434c-ba73-3813499010b7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:38.571555 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9be5065e-b602-434c-ba73-3813499010b7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:38.571981 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9be5065e-b602-434c-ba73-3813499010b7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:38.603971 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-de98cc65-8395-4142-b83e-0316193b1fd2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:38.604723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-de98cc65-8395-4142-b83e-0316193b1fd2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:38.605181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9be5065e-b602-434c-ba73-3813499010b7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048826 Sep 08 14:02:38.605764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 745/1384] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:38 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:38.610679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-de98cc65-8395-4142-b83e-0316193b1fd2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:38.611161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-de98cc65-8395-4142-b83e-0316193b1fd2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:38.611595 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-de98cc65-8395-4142-b83e-0316193b1fd2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:38.612294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-de98cc65-8395-4142-b83e-0316193b1fd2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:38.612745 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-de98cc65-8395-4142-b83e-0316193b1fd2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:38.613166 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-de98cc65-8395-4142-b83e-0316193b1fd2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:38.689759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-de98cc65-8395-4142-b83e-0316193b1fd2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:38.699233 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-de98cc65-8395-4142-b83e-0316193b1fd2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.099750 Sep 08 14:02:38.699808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 640/1385] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:38 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:39.094290 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6dedb674-0f98-410f-8989-8304669783a3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:39.096479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6dedb674-0f98-410f-8989-8304669783a3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:39.102847 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6dedb674-0f98-410f-8989-8304669783a3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:39.103347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6dedb674-0f98-410f-8989-8304669783a3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:39.103783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6dedb674-0f98-410f-8989-8304669783a3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:39.104519 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6dedb674-0f98-410f-8989-8304669783a3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:39.104951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6dedb674-0f98-410f-8989-8304669783a3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:39.105380 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6dedb674-0f98-410f-8989-8304669783a3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:39.137248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6dedb674-0f98-410f-8989-8304669783a3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046984 Sep 08 14:02:39.138047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 746/1386] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:39 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:39.373536 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f599fc95-698c-4d0d-b720-fe9deedd665f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:39.374060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f599fc95-698c-4d0d-b720-fe9deedd665f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:39.381499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f599fc95-698c-4d0d-b720-fe9deedd665f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:39.381993 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f599fc95-698c-4d0d-b720-fe9deedd665f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:39.382461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f599fc95-698c-4d0d-b720-fe9deedd665f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:39.383166 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f599fc95-698c-4d0d-b720-fe9deedd665f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:39.383619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f599fc95-698c-4d0d-b720-fe9deedd665f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:39.384047 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f599fc95-698c-4d0d-b720-fe9deedd665f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:39.446095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-34a347da-7ce4-4a55-b6c3-99a20c71bbe8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:39.448678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-34a347da-7ce4-4a55-b6c3-99a20c71bbe8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:39.456624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-34a347da-7ce4-4a55-b6c3-99a20c71bbe8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:39.457113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-34a347da-7ce4-4a55-b6c3-99a20c71bbe8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:39.457550 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-34a347da-7ce4-4a55-b6c3-99a20c71bbe8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:39.458289 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-34a347da-7ce4-4a55-b6c3-99a20c71bbe8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:39.458753 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-34a347da-7ce4-4a55-b6c3-99a20c71bbe8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:39.459181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-34a347da-7ce4-4a55-b6c3-99a20c71bbe8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:39.491962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-34a347da-7ce4-4a55-b6c3-99a20c71bbe8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051596 Sep 08 14:02:39.492698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 747/1387] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:39 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:39.536233 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f599fc95-698c-4d0d-b720-fe9deedd665f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:39.544819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f599fc95-698c-4d0d-b720-fe9deedd665f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.175281 Sep 08 14:02:39.545390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 641/1388] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:39 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:39.620913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1b8eff65-9eeb-4dea-b56c-22ae18f2c8c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:39.620913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1b8eff65-9eeb-4dea-b56c-22ae18f2c8c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:39.629891 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1b8eff65-9eeb-4dea-b56c-22ae18f2c8c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:39.630690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1b8eff65-9eeb-4dea-b56c-22ae18f2c8c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:39.630927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1b8eff65-9eeb-4dea-b56c-22ae18f2c8c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:39.631635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1b8eff65-9eeb-4dea-b56c-22ae18f2c8c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:39.632096 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1b8eff65-9eeb-4dea-b56c-22ae18f2c8c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:39.632529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1b8eff65-9eeb-4dea-b56c-22ae18f2c8c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:39.662925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1b8eff65-9eeb-4dea-b56c-22ae18f2c8c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046249 Sep 08 14:02:39.663490 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 748/1389] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:39 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:39.722770 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6ba81607-be3a-4aa4-b177-44c5e4fdb757 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:39.723369 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6ba81607-be3a-4aa4-b177-44c5e4fdb757 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:39.730054 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6ba81607-be3a-4aa4-b177-44c5e4fdb757 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:39.730566 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6ba81607-be3a-4aa4-b177-44c5e4fdb757 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:39.730995 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6ba81607-be3a-4aa4-b177-44c5e4fdb757 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:39.731684 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6ba81607-be3a-4aa4-b177-44c5e4fdb757 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:39.732126 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6ba81607-be3a-4aa4-b177-44c5e4fdb757 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:39.732593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6ba81607-be3a-4aa4-b177-44c5e4fdb757 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:39.796305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6ba81607-be3a-4aa4-b177-44c5e4fdb757 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:39.804128 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6ba81607-be3a-4aa4-b177-44c5e4fdb757 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.085212 Sep 08 14:02:39.804685 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 642/1390] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:39 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:40.151187 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e8198247-f780-4925-8204-8cd98a32e8df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:40.153486 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e8198247-f780-4925-8204-8cd98a32e8df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:40.160751 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e8198247-f780-4925-8204-8cd98a32e8df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:40.161288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e8198247-f780-4925-8204-8cd98a32e8df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:40.161884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e8198247-f780-4925-8204-8cd98a32e8df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:40.162642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e8198247-f780-4925-8204-8cd98a32e8df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:40.163149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e8198247-f780-4925-8204-8cd98a32e8df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:40.163613 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e8198247-f780-4925-8204-8cd98a32e8df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:40.207704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e8198247-f780-4925-8204-8cd98a32e8df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061762 Sep 08 14:02:40.208920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 749/1391] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:40 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:40.511014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-52fa3bcd-92cc-4754-a030-1fbafc328b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:40.513278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-52fa3bcd-92cc-4754-a030-1fbafc328b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:40.520004 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-52fa3bcd-92cc-4754-a030-1fbafc328b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:40.520484 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-52fa3bcd-92cc-4754-a030-1fbafc328b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:40.520914 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-52fa3bcd-92cc-4754-a030-1fbafc328b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:40.521602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-52fa3bcd-92cc-4754-a030-1fbafc328b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:40.522043 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-52fa3bcd-92cc-4754-a030-1fbafc328b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:40.522483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-52fa3bcd-92cc-4754-a030-1fbafc328b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:40.555725 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-52fa3bcd-92cc-4754-a030-1fbafc328b0b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050327 Sep 08 14:02:40.556320 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 643/1392] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:40 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:40.569046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2abb8a02-e328-45c4-8bb7-501d43f52027 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:40.570153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2abb8a02-e328-45c4-8bb7-501d43f52027 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:40.576288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2abb8a02-e328-45c4-8bb7-501d43f52027 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:40.576884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2abb8a02-e328-45c4-8bb7-501d43f52027 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:40.577326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2abb8a02-e328-45c4-8bb7-501d43f52027 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:40.578032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2abb8a02-e328-45c4-8bb7-501d43f52027 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:40.578512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2abb8a02-e328-45c4-8bb7-501d43f52027 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:40.578936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2abb8a02-e328-45c4-8bb7-501d43f52027 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:40.684842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bfa51953-12cf-491a-b1d3-773d9bcce02b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:40.691992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bfa51953-12cf-491a-b1d3-773d9bcce02b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:40.703909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bfa51953-12cf-491a-b1d3-773d9bcce02b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:40.704811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bfa51953-12cf-491a-b1d3-773d9bcce02b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:40.705704 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bfa51953-12cf-491a-b1d3-773d9bcce02b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:40.707158 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bfa51953-12cf-491a-b1d3-773d9bcce02b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:40.708098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bfa51953-12cf-491a-b1d3-773d9bcce02b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:40.708799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bfa51953-12cf-491a-b1d3-773d9bcce02b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:40.756495 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2abb8a02-e328-45c4-8bb7-501d43f52027 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:40.761880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bfa51953-12cf-491a-b1d3-773d9bcce02b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.090239 Sep 08 14:02:40.762823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 644/1393] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:40 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:02:40.770297 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2abb8a02-e328-45c4-8bb7-501d43f52027 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.205626 Sep 08 14:02:40.771078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 750/1394] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:40 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:40.828065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b31467e0-a372-4646-a6df-4e0f71ad6e15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:40.828878 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b31467e0-a372-4646-a6df-4e0f71ad6e15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:40.836704 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b31467e0-a372-4646-a6df-4e0f71ad6e15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:40.837197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b31467e0-a372-4646-a6df-4e0f71ad6e15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:40.837629 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b31467e0-a372-4646-a6df-4e0f71ad6e15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:40.838388 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b31467e0-a372-4646-a6df-4e0f71ad6e15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:40.838844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b31467e0-a372-4646-a6df-4e0f71ad6e15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:40.839263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b31467e0-a372-4646-a6df-4e0f71ad6e15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:41.162700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b31467e0-a372-4646-a6df-4e0f71ad6e15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:41.170722 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b31467e0-a372-4646-a6df-4e0f71ad6e15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.347034 Sep 08 14:02:41.171299 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 645/1395] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:40 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:41.221135 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-385e6753-5c16-4ac5-b0e3-930a304b11bd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:41.223368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-385e6753-5c16-4ac5-b0e3-930a304b11bd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:41.230244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-385e6753-5c16-4ac5-b0e3-930a304b11bd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:41.230735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-385e6753-5c16-4ac5-b0e3-930a304b11bd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:41.231165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-385e6753-5c16-4ac5-b0e3-930a304b11bd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:41.231867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-385e6753-5c16-4ac5-b0e3-930a304b11bd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:41.232316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-385e6753-5c16-4ac5-b0e3-930a304b11bd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:41.232738 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-385e6753-5c16-4ac5-b0e3-930a304b11bd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:41.264969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-385e6753-5c16-4ac5-b0e3-930a304b11bd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047828 Sep 08 14:02:41.265528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 751/1396] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:41 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:41.569580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-13a9f93c-da58-4554-b24b-9f93078a8ee6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:41.571687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-13a9f93c-da58-4554-b24b-9f93078a8ee6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:41.578224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-13a9f93c-da58-4554-b24b-9f93078a8ee6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:41.578704 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-13a9f93c-da58-4554-b24b-9f93078a8ee6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:41.579134 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-13a9f93c-da58-4554-b24b-9f93078a8ee6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:41.579812 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-13a9f93c-da58-4554-b24b-9f93078a8ee6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:41.580259 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-13a9f93c-da58-4554-b24b-9f93078a8ee6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:41.580671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-13a9f93c-da58-4554-b24b-9f93078a8ee6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:41.611989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-13a9f93c-da58-4554-b24b-9f93078a8ee6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046457 Sep 08 14:02:41.612612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 646/1397] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:41 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:41.775135 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-69d3fd0f-39ac-419a-96b4-cedcd446414f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:41.777294 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-69d3fd0f-39ac-419a-96b4-cedcd446414f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:41.783822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69d3fd0f-39ac-419a-96b4-cedcd446414f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:41.784269 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69d3fd0f-39ac-419a-96b4-cedcd446414f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:41.784701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69d3fd0f-39ac-419a-96b4-cedcd446414f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:41.785388 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69d3fd0f-39ac-419a-96b4-cedcd446414f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:41.785836 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69d3fd0f-39ac-419a-96b4-cedcd446414f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:41.786316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69d3fd0f-39ac-419a-96b4-cedcd446414f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:41.794265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fe1bae45-7a9b-414e-bf3c-e9d4af161658 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:41.794879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fe1bae45-7a9b-414e-bf3c-e9d4af161658 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:41.800931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fe1bae45-7a9b-414e-bf3c-e9d4af161658 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:41.801423 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fe1bae45-7a9b-414e-bf3c-e9d4af161658 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:41.801857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fe1bae45-7a9b-414e-bf3c-e9d4af161658 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:41.802589 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fe1bae45-7a9b-414e-bf3c-e9d4af161658 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:41.803046 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fe1bae45-7a9b-414e-bf3c-e9d4af161658 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:41.803475 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fe1bae45-7a9b-414e-bf3c-e9d4af161658 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:41.818608 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-69d3fd0f-39ac-419a-96b4-cedcd446414f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047951 Sep 08 14:02:41.819421 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 752/1398] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:41 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:41.944853 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-fe1bae45-7a9b-414e-bf3c-e9d4af161658 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:41.953022 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fe1bae45-7a9b-414e-bf3c-e9d4af161658 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.162604 Sep 08 14:02:41.953606 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 647/1399] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:41 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:42.195966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c930a169-8f56-4d6e-8bf0-37097116e642 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:42.196502 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c930a169-8f56-4d6e-8bf0-37097116e642 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:42.202769 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c930a169-8f56-4d6e-8bf0-37097116e642 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:42.203263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c930a169-8f56-4d6e-8bf0-37097116e642 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:42.203707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c930a169-8f56-4d6e-8bf0-37097116e642 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:42.204443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c930a169-8f56-4d6e-8bf0-37097116e642 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:42.204897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c930a169-8f56-4d6e-8bf0-37097116e642 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:42.205326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c930a169-8f56-4d6e-8bf0-37097116e642 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:42.272348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c930a169-8f56-4d6e-8bf0-37097116e642 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:42.277396 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b66cfab2-175d-45ee-b7ec-efe230d0bc3c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:42.279594 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b66cfab2-175d-45ee-b7ec-efe230d0bc3c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:42.281425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c930a169-8f56-4d6e-8bf0-37097116e642 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.089997 Sep 08 14:02:42.286018 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 753/1400] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:42 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:42.286228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b66cfab2-175d-45ee-b7ec-efe230d0bc3c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:42.286425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b66cfab2-175d-45ee-b7ec-efe230d0bc3c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:42.287153 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b66cfab2-175d-45ee-b7ec-efe230d0bc3c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:42.288170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b66cfab2-175d-45ee-b7ec-efe230d0bc3c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:42.288905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b66cfab2-175d-45ee-b7ec-efe230d0bc3c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:42.289057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b66cfab2-175d-45ee-b7ec-efe230d0bc3c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:42.323473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b66cfab2-175d-45ee-b7ec-efe230d0bc3c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050384 Sep 08 14:02:42.324047 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 648/1401] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:42 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:42.624933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5a1451c9-321b-49e7-8b30-c137f8b63034 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:42.627483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5a1451c9-321b-49e7-8b30-c137f8b63034 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:42.633644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a1451c9-321b-49e7-8b30-c137f8b63034 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:42.634161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a1451c9-321b-49e7-8b30-c137f8b63034 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:42.634601 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a1451c9-321b-49e7-8b30-c137f8b63034 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:42.635301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a1451c9-321b-49e7-8b30-c137f8b63034 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:42.635756 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a1451c9-321b-49e7-8b30-c137f8b63034 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:42.636181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a1451c9-321b-49e7-8b30-c137f8b63034 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:42.668887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5a1451c9-321b-49e7-8b30-c137f8b63034 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047669 Sep 08 14:02:42.669717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 754/1402] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:42 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:42.835553 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-000570ff-1461-4b10-a9a5-db4a66263335 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:42.837867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-000570ff-1461-4b10-a9a5-db4a66263335 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:42.847666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-000570ff-1461-4b10-a9a5-db4a66263335 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:42.848410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-000570ff-1461-4b10-a9a5-db4a66263335 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:42.849659 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-000570ff-1461-4b10-a9a5-db4a66263335 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:42.850784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-000570ff-1461-4b10-a9a5-db4a66263335 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:42.851494 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-000570ff-1461-4b10-a9a5-db4a66263335 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:42.852168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-000570ff-1461-4b10-a9a5-db4a66263335 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:42.883646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-000570ff-1461-4b10-a9a5-db4a66263335 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054189 Sep 08 14:02:42.884203 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 649/1403] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:42 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:42.977479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d98c571d-c103-42cf-b4be-0ef96e3b145e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:42.978386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d98c571d-c103-42cf-b4be-0ef96e3b145e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:42.984888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d98c571d-c103-42cf-b4be-0ef96e3b145e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:42.985373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d98c571d-c103-42cf-b4be-0ef96e3b145e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:42.985919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d98c571d-c103-42cf-b4be-0ef96e3b145e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:42.986642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d98c571d-c103-42cf-b4be-0ef96e3b145e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:42.987097 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d98c571d-c103-42cf-b4be-0ef96e3b145e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:42.987524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d98c571d-c103-42cf-b4be-0ef96e3b145e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:43.127327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d98c571d-c103-42cf-b4be-0ef96e3b145e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:43.136137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d98c571d-c103-42cf-b4be-0ef96e3b145e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.163142 Sep 08 14:02:43.136828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 755/1404] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:42 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:43.319219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cae51f60-8613-45f5-a556-025543cdc078 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:43.319743 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cae51f60-8613-45f5-a556-025543cdc078 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:43.325732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cae51f60-8613-45f5-a556-025543cdc078 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:43.326243 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cae51f60-8613-45f5-a556-025543cdc078 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:43.326680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cae51f60-8613-45f5-a556-025543cdc078 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:43.327375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cae51f60-8613-45f5-a556-025543cdc078 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:43.327831 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cae51f60-8613-45f5-a556-025543cdc078 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:43.328255 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cae51f60-8613-45f5-a556-025543cdc078 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:43.336617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-06a88be0-74c9-4973-bedf-448b96acc3da tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:43.338877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-06a88be0-74c9-4973-bedf-448b96acc3da tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:43.344917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06a88be0-74c9-4973-bedf-448b96acc3da tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:43.345405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06a88be0-74c9-4973-bedf-448b96acc3da tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:43.345844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06a88be0-74c9-4973-bedf-448b96acc3da tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:43.346571 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06a88be0-74c9-4973-bedf-448b96acc3da tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:43.347027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06a88be0-74c9-4973-bedf-448b96acc3da tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:43.347458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06a88be0-74c9-4973-bedf-448b96acc3da tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:43.379208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-06a88be0-74c9-4973-bedf-448b96acc3da tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047189 Sep 08 14:02:43.380317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 756/1405] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:43 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:43.391448 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-cae51f60-8613-45f5-a556-025543cdc078 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:43.399926 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cae51f60-8613-45f5-a556-025543cdc078 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.085372 Sep 08 14:02:43.400506 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 650/1406] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:43 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:43.682793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c6c21682-b3f5-4103-a333-2528dfadc10a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:43.684944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c6c21682-b3f5-4103-a333-2528dfadc10a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:43.691019 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c6c21682-b3f5-4103-a333-2528dfadc10a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:43.691504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c6c21682-b3f5-4103-a333-2528dfadc10a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:43.691943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c6c21682-b3f5-4103-a333-2528dfadc10a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:43.693256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c6c21682-b3f5-4103-a333-2528dfadc10a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:43.693721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c6c21682-b3f5-4103-a333-2528dfadc10a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:43.694286 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c6c21682-b3f5-4103-a333-2528dfadc10a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:43.723106 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c6c21682-b3f5-4103-a333-2528dfadc10a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044513 Sep 08 14:02:43.723662 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 757/1407] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:43 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:43.896482 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c006d2ec-9adf-40bc-8ea4-f4e0554e3229 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:43.898711 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c006d2ec-9adf-40bc-8ea4-f4e0554e3229 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:43.905068 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c006d2ec-9adf-40bc-8ea4-f4e0554e3229 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:43.905540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c006d2ec-9adf-40bc-8ea4-f4e0554e3229 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:43.905967 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c006d2ec-9adf-40bc-8ea4-f4e0554e3229 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:43.906682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c006d2ec-9adf-40bc-8ea4-f4e0554e3229 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:43.907129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c006d2ec-9adf-40bc-8ea4-f4e0554e3229 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:43.907550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c006d2ec-9adf-40bc-8ea4-f4e0554e3229 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:43.938825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c006d2ec-9adf-40bc-8ea4-f4e0554e3229 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046375 Sep 08 14:02:43.939367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 651/1408] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:43 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:44.160350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e601769c-fafa-4c6e-a83d-f160629580ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:44.161400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e601769c-fafa-4c6e-a83d-f160629580ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:44.167941 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e601769c-fafa-4c6e-a83d-f160629580ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:44.168418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e601769c-fafa-4c6e-a83d-f160629580ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:44.168847 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e601769c-fafa-4c6e-a83d-f160629580ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:44.169539 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e601769c-fafa-4c6e-a83d-f160629580ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:44.169989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e601769c-fafa-4c6e-a83d-f160629580ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:44.170440 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e601769c-fafa-4c6e-a83d-f160629580ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:44.304698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e601769c-fafa-4c6e-a83d-f160629580ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:44.312568 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e601769c-fafa-4c6e-a83d-f160629580ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.156187 Sep 08 14:02:44.313129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 758/1409] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:44 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:44.392085 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-aa59e190-c5fa-4015-99ad-6e96564c3c9b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:44.394257 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-aa59e190-c5fa-4015-99ad-6e96564c3c9b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:44.401090 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa59e190-c5fa-4015-99ad-6e96564c3c9b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:44.401566 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa59e190-c5fa-4015-99ad-6e96564c3c9b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:44.401997 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa59e190-c5fa-4015-99ad-6e96564c3c9b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:44.402713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa59e190-c5fa-4015-99ad-6e96564c3c9b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:44.403157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa59e190-c5fa-4015-99ad-6e96564c3c9b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:44.403576 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa59e190-c5fa-4015-99ad-6e96564c3c9b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:44.422500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-83a08486-fafc-4fd0-905a-5cbfa35c67ab tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:44.423033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-83a08486-fafc-4fd0-905a-5cbfa35c67ab tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:44.429244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-83a08486-fafc-4fd0-905a-5cbfa35c67ab tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:44.429740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-83a08486-fafc-4fd0-905a-5cbfa35c67ab tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:44.430365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-83a08486-fafc-4fd0-905a-5cbfa35c67ab tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:44.430928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-83a08486-fafc-4fd0-905a-5cbfa35c67ab tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:44.431460 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-83a08486-fafc-4fd0-905a-5cbfa35c67ab tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:44.431893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-83a08486-fafc-4fd0-905a-5cbfa35c67ab tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:44.433481 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-aa59e190-c5fa-4015-99ad-6e96564c3c9b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045292 Sep 08 14:02:44.434045 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 652/1410] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:44 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:44.447404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:44.499265 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-83a08486-fafc-4fd0-905a-5cbfa35c67ab tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:44.507443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-83a08486-fafc-4fd0-905a-5cbfa35c67ab tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.088888 Sep 08 14:02:44.508020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 759/1411] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:44 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:44.740734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c03fcd0d-65ae-4b3a-b12a-d1449f09ae46 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:44.744385 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c03fcd0d-65ae-4b3a-b12a-d1449f09ae46 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:44.752911 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c03fcd0d-65ae-4b3a-b12a-d1449f09ae46 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:44.753399 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c03fcd0d-65ae-4b3a-b12a-d1449f09ae46 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:44.753835 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c03fcd0d-65ae-4b3a-b12a-d1449f09ae46 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:44.755064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c03fcd0d-65ae-4b3a-b12a-d1449f09ae46 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:44.755569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c03fcd0d-65ae-4b3a-b12a-d1449f09ae46 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:44.755975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c03fcd0d-65ae-4b3a-b12a-d1449f09ae46 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:44.785670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c03fcd0d-65ae-4b3a-b12a-d1449f09ae46 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052823 Sep 08 14:02:44.786291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 653/1412] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:44 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:44.952046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2e399ae8-ec5c-4516-895b-efdbabdc24ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:44.954178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2e399ae8-ec5c-4516-895b-efdbabdc24ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:44.960861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e399ae8-ec5c-4516-895b-efdbabdc24ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:44.961340 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e399ae8-ec5c-4516-895b-efdbabdc24ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:44.961775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e399ae8-ec5c-4516-895b-efdbabdc24ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:44.962486 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e399ae8-ec5c-4516-895b-efdbabdc24ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:44.962935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e399ae8-ec5c-4516-895b-efdbabdc24ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:44.963353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e399ae8-ec5c-4516-895b-efdbabdc24ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:44.993463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2e399ae8-ec5c-4516-895b-efdbabdc24ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045281 Sep 08 14:02:44.993935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 760/1413] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:44 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:45.336019 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-58125859-0c79-4e1c-bdad-43400755b5c4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:45.336629 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-58125859-0c79-4e1c-bdad-43400755b5c4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:45.343411 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-58125859-0c79-4e1c-bdad-43400755b5c4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:45.343902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-58125859-0c79-4e1c-bdad-43400755b5c4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:45.344337 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-58125859-0c79-4e1c-bdad-43400755b5c4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:45.345033 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-58125859-0c79-4e1c-bdad-43400755b5c4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:45.345481 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-58125859-0c79-4e1c-bdad-43400755b5c4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:45.345898 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-58125859-0c79-4e1c-bdad-43400755b5c4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:45.445070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ccf80dbc-f754-4007-8c3c-866f9fb82bcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:45.447411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ccf80dbc-f754-4007-8c3c-866f9fb82bcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:45.453911 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ccf80dbc-f754-4007-8c3c-866f9fb82bcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:45.454437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ccf80dbc-f754-4007-8c3c-866f9fb82bcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:45.454881 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ccf80dbc-f754-4007-8c3c-866f9fb82bcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:45.455587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ccf80dbc-f754-4007-8c3c-866f9fb82bcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:45.456036 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ccf80dbc-f754-4007-8c3c-866f9fb82bcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:45.456494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ccf80dbc-f754-4007-8c3c-866f9fb82bcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:45.484586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-58125859-0c79-4e1c-bdad-43400755b5c4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:45.493588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-58125859-0c79-4e1c-bdad-43400755b5c4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.161422 Sep 08 14:02:45.494517 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 654/1414] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:45 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:45.495943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ccf80dbc-f754-4007-8c3c-866f9fb82bcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054884 Sep 08 14:02:45.496544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 761/1415] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:45 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:45.531775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7b9b17d4-e742-455c-b829-4ff11baef7ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:45.532370 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7b9b17d4-e742-455c-b829-4ff11baef7ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:45.538831 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b9b17d4-e742-455c-b829-4ff11baef7ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:45.539308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b9b17d4-e742-455c-b829-4ff11baef7ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:45.539735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b9b17d4-e742-455c-b829-4ff11baef7ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:45.540414 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b9b17d4-e742-455c-b829-4ff11baef7ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:45.540856 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b9b17d4-e742-455c-b829-4ff11baef7ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:45.541270 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b9b17d4-e742-455c-b829-4ff11baef7ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:45.560228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:45.607407 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7b9b17d4-e742-455c-b829-4ff11baef7ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:45.615907 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7b9b17d4-e742-455c-b829-4ff11baef7ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.087929 Sep 08 14:02:45.616475 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 655/1416] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:45 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:45.797408 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-41415526-3a94-43ef-8395-6c58f79cfa1a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:45.799601 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-41415526-3a94-43ef-8395-6c58f79cfa1a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:45.805533 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-41415526-3a94-43ef-8395-6c58f79cfa1a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:45.806015 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-41415526-3a94-43ef-8395-6c58f79cfa1a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:45.806482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-41415526-3a94-43ef-8395-6c58f79cfa1a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:45.807172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-41415526-3a94-43ef-8395-6c58f79cfa1a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:45.807620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-41415526-3a94-43ef-8395-6c58f79cfa1a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:45.808044 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-41415526-3a94-43ef-8395-6c58f79cfa1a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:45.837940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-41415526-3a94-43ef-8395-6c58f79cfa1a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044531 Sep 08 14:02:45.838523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 762/1417] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:45 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:46.006496 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4a571ab5-ba66-4459-8d73-72691bac5b5d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:46.008679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4a571ab5-ba66-4459-8d73-72691bac5b5d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:46.014843 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a571ab5-ba66-4459-8d73-72691bac5b5d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:46.015317 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a571ab5-ba66-4459-8d73-72691bac5b5d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:46.015781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a571ab5-ba66-4459-8d73-72691bac5b5d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:46.016516 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a571ab5-ba66-4459-8d73-72691bac5b5d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:46.016994 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a571ab5-ba66-4459-8d73-72691bac5b5d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:46.017416 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a571ab5-ba66-4459-8d73-72691bac5b5d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:46.048421 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4a571ab5-ba66-4459-8d73-72691bac5b5d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046488 Sep 08 14:02:46.049028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 656/1418] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:46 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:46.510246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-abc74e47-e63d-4209-a8b6-e3617a3669aa tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:46.512444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-abc74e47-e63d-4209-a8b6-e3617a3669aa tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:46.517971 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-38f3c8e7-4ae7-4788-9683-11023b1b37f5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:46.520675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-38f3c8e7-4ae7-4788-9683-11023b1b37f5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:46.520804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-abc74e47-e63d-4209-a8b6-e3617a3669aa tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:46.520804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-abc74e47-e63d-4209-a8b6-e3617a3669aa tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:46.520804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-abc74e47-e63d-4209-a8b6-e3617a3669aa tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:46.520804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-abc74e47-e63d-4209-a8b6-e3617a3669aa tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:46.521210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-abc74e47-e63d-4209-a8b6-e3617a3669aa tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:46.521926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-abc74e47-e63d-4209-a8b6-e3617a3669aa tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:46.531905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38f3c8e7-4ae7-4788-9683-11023b1b37f5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:46.532384 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38f3c8e7-4ae7-4788-9683-11023b1b37f5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:46.532819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38f3c8e7-4ae7-4788-9683-11023b1b37f5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:46.533547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38f3c8e7-4ae7-4788-9683-11023b1b37f5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:46.534010 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38f3c8e7-4ae7-4788-9683-11023b1b37f5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:46.534526 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38f3c8e7-4ae7-4788-9683-11023b1b37f5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:46.572668 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-abc74e47-e63d-4209-a8b6-e3617a3669aa tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067246 Sep 08 14:02:46.573261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 763/1419] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:46 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:46.639435 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f65d3c18-b38b-4edd-8564-9d033bf0cb15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:46.640056 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f65d3c18-b38b-4edd-8564-9d033bf0cb15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:46.646931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f65d3c18-b38b-4edd-8564-9d033bf0cb15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:46.647650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f65d3c18-b38b-4edd-8564-9d033bf0cb15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:46.648038 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f65d3c18-b38b-4edd-8564-9d033bf0cb15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:46.650262 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f65d3c18-b38b-4edd-8564-9d033bf0cb15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:46.650262 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f65d3c18-b38b-4edd-8564-9d033bf0cb15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:46.650262 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f65d3c18-b38b-4edd-8564-9d033bf0cb15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:46.711450 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-38f3c8e7-4ae7-4788-9683-11023b1b37f5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:46.720894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f65d3c18-b38b-4edd-8564-9d033bf0cb15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:46.723084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-38f3c8e7-4ae7-4788-9683-11023b1b37f5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.208625 Sep 08 14:02:46.729633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f65d3c18-b38b-4edd-8564-9d033bf0cb15 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.094213 Sep 08 14:02:46.730595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 764/1420] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:46 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:46.730774 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 657/1421] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:46 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:46.853784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-47b1633b-8794-49e4-9341-e07c94841b72 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:46.856289 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-47b1633b-8794-49e4-9341-e07c94841b72 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:46.862664 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47b1633b-8794-49e4-9341-e07c94841b72 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:46.863159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47b1633b-8794-49e4-9341-e07c94841b72 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:46.863598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47b1633b-8794-49e4-9341-e07c94841b72 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:46.864451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47b1633b-8794-49e4-9341-e07c94841b72 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:46.865417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47b1633b-8794-49e4-9341-e07c94841b72 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:46.865848 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47b1633b-8794-49e4-9341-e07c94841b72 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:46.896353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-47b1633b-8794-49e4-9341-e07c94841b72 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049053 Sep 08 14:02:46.896922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 765/1422] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:46 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:47.061276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5ea3c79c-dfad-4dd7-90d8-4e07d4861585 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:47.063463 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5ea3c79c-dfad-4dd7-90d8-4e07d4861585 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:47.069730 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5ea3c79c-dfad-4dd7-90d8-4e07d4861585 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:47.070257 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5ea3c79c-dfad-4dd7-90d8-4e07d4861585 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:47.070688 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5ea3c79c-dfad-4dd7-90d8-4e07d4861585 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:47.071378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5ea3c79c-dfad-4dd7-90d8-4e07d4861585 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:47.071818 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5ea3c79c-dfad-4dd7-90d8-4e07d4861585 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:47.072230 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5ea3c79c-dfad-4dd7-90d8-4e07d4861585 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:47.104979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5ea3c79c-dfad-4dd7-90d8-4e07d4861585 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047692 Sep 08 14:02:47.105441 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 658/1423] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:47 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:47.586037 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-aae538c4-33f0-4a6c-baa7-1fc961afbc0c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:47.588315 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-aae538c4-33f0-4a6c-baa7-1fc961afbc0c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:47.595145 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aae538c4-33f0-4a6c-baa7-1fc961afbc0c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:47.595643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aae538c4-33f0-4a6c-baa7-1fc961afbc0c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:47.596087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aae538c4-33f0-4a6c-baa7-1fc961afbc0c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:47.596798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aae538c4-33f0-4a6c-baa7-1fc961afbc0c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:47.597323 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aae538c4-33f0-4a6c-baa7-1fc961afbc0c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:47.597755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aae538c4-33f0-4a6c-baa7-1fc961afbc0c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:47.628736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-aae538c4-33f0-4a6c-baa7-1fc961afbc0c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047375 Sep 08 14:02:47.629308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 766/1424] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:47 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:47.748344 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-628fc6b5-8882-4293-85f3-509f98757908 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:47.749750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-628fc6b5-8882-4293-85f3-509f98757908 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:47.756658 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-628fc6b5-8882-4293-85f3-509f98757908 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:47.757144 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-628fc6b5-8882-4293-85f3-509f98757908 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:47.757577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-628fc6b5-8882-4293-85f3-509f98757908 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:47.758306 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-628fc6b5-8882-4293-85f3-509f98757908 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:47.758773 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-628fc6b5-8882-4293-85f3-509f98757908 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:47.758921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5497fb02-0011-478d-b1ff-860ddaed611f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:47.759182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-628fc6b5-8882-4293-85f3-509f98757908 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:47.759405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5497fb02-0011-478d-b1ff-860ddaed611f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:47.765219 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5497fb02-0011-478d-b1ff-860ddaed611f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:47.765709 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5497fb02-0011-478d-b1ff-860ddaed611f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:47.766183 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5497fb02-0011-478d-b1ff-860ddaed611f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:47.766887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5497fb02-0011-478d-b1ff-860ddaed611f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:47.767346 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5497fb02-0011-478d-b1ff-860ddaed611f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:47.767777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5497fb02-0011-478d-b1ff-860ddaed611f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:47.838547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5497fb02-0011-478d-b1ff-860ddaed611f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:47.846458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5497fb02-0011-478d-b1ff-860ddaed611f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.092359 Sep 08 14:02:47.847028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 767/1425] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:47 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:47.897858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-628fc6b5-8882-4293-85f3-509f98757908 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:47.905777 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-628fc6b5-8882-4293-85f3-509f98757908 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.161615 Sep 08 14:02:47.906381 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 659/1426] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:47 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:47.908951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e558c239-c8ad-4e30-b8dc-7fbbcb07e88a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:47.911348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e558c239-c8ad-4e30-b8dc-7fbbcb07e88a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:47.917289 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e558c239-c8ad-4e30-b8dc-7fbbcb07e88a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:47.917777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e558c239-c8ad-4e30-b8dc-7fbbcb07e88a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:47.918245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e558c239-c8ad-4e30-b8dc-7fbbcb07e88a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:47.918953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e558c239-c8ad-4e30-b8dc-7fbbcb07e88a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:47.919420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e558c239-c8ad-4e30-b8dc-7fbbcb07e88a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:47.919849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e558c239-c8ad-4e30-b8dc-7fbbcb07e88a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:47.950043 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e558c239-c8ad-4e30-b8dc-7fbbcb07e88a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045294 Sep 08 14:02:47.950636 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 768/1427] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:47 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:48.119940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-695a701f-cd56-4675-b500-7b0fc0a73443 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:48.122064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-695a701f-cd56-4675-b500-7b0fc0a73443 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:48.128365 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-695a701f-cd56-4675-b500-7b0fc0a73443 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:48.128839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-695a701f-cd56-4675-b500-7b0fc0a73443 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:48.129276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-695a701f-cd56-4675-b500-7b0fc0a73443 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:48.130677 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-695a701f-cd56-4675-b500-7b0fc0a73443 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:48.131152 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-695a701f-cd56-4675-b500-7b0fc0a73443 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:48.131569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-695a701f-cd56-4675-b500-7b0fc0a73443 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:48.161091 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-695a701f-cd56-4675-b500-7b0fc0a73443 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045103 Sep 08 14:02:48.161650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 660/1428] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:48 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:48.641070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7bf880d3-b588-4273-a3c6-3c4df1c00cf8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:48.643259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7bf880d3-b588-4273-a3c6-3c4df1c00cf8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:48.650920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7bf880d3-b588-4273-a3c6-3c4df1c00cf8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:48.651409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7bf880d3-b588-4273-a3c6-3c4df1c00cf8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:48.651848 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7bf880d3-b588-4273-a3c6-3c4df1c00cf8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:48.652549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7bf880d3-b588-4273-a3c6-3c4df1c00cf8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:48.653004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7bf880d3-b588-4273-a3c6-3c4df1c00cf8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:48.653432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7bf880d3-b588-4273-a3c6-3c4df1c00cf8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:48.684541 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7bf880d3-b588-4273-a3c6-3c4df1c00cf8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047430 Sep 08 14:02:48.685104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 769/1429] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:48 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:48.869812 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4a56165e-2f09-4ca4-97d8-0e2ea54bf4c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:48.870421 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4a56165e-2f09-4ca4-97d8-0e2ea54bf4c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:48.877119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a56165e-2f09-4ca4-97d8-0e2ea54bf4c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:48.877601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a56165e-2f09-4ca4-97d8-0e2ea54bf4c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:48.878034 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a56165e-2f09-4ca4-97d8-0e2ea54bf4c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:48.878744 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a56165e-2f09-4ca4-97d8-0e2ea54bf4c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:48.879546 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a56165e-2f09-4ca4-97d8-0e2ea54bf4c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:48.880413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a56165e-2f09-4ca4-97d8-0e2ea54bf4c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:48.929697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8bb1b4b0-b2d6-4e5f-9956-fc0d3e0f403b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:48.930241 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8bb1b4b0-b2d6-4e5f-9956-fc0d3e0f403b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:48.936199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8bb1b4b0-b2d6-4e5f-9956-fc0d3e0f403b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:48.936681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8bb1b4b0-b2d6-4e5f-9956-fc0d3e0f403b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:48.937113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8bb1b4b0-b2d6-4e5f-9956-fc0d3e0f403b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:48.937805 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8bb1b4b0-b2d6-4e5f-9956-fc0d3e0f403b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:48.938282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8bb1b4b0-b2d6-4e5f-9956-fc0d3e0f403b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:48.938707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8bb1b4b0-b2d6-4e5f-9956-fc0d3e0f403b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:48.946972 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4a56165e-2f09-4ca4-97d8-0e2ea54bf4c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:48.955017 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4a56165e-2f09-4ca4-97d8-0e2ea54bf4c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.089072 Sep 08 14:02:48.955777 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 661/1430] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:48 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:48.962415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-21253f2e-9e58-417d-a62b-38a5be83802d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:48.964612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-21253f2e-9e58-417d-a62b-38a5be83802d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:48.970908 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21253f2e-9e58-417d-a62b-38a5be83802d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:48.971413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21253f2e-9e58-417d-a62b-38a5be83802d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:48.978960 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21253f2e-9e58-417d-a62b-38a5be83802d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:48.978960 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21253f2e-9e58-417d-a62b-38a5be83802d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:48.978960 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21253f2e-9e58-417d-a62b-38a5be83802d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:48.978960 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21253f2e-9e58-417d-a62b-38a5be83802d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:49.004534 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-21253f2e-9e58-417d-a62b-38a5be83802d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046011 Sep 08 14:02:49.010119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 662/1431] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:48 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:49.083390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8bb1b4b0-b2d6-4e5f-9956-fc0d3e0f403b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:49.091411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8bb1b4b0-b2d6-4e5f-9956-fc0d3e0f403b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.166272 Sep 08 14:02:49.091780 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 770/1432] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:48 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:49.173143 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d04b1221-b884-4816-9304-39defdddf814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:49.175310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d04b1221-b884-4816-9304-39defdddf814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:49.182620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d04b1221-b884-4816-9304-39defdddf814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:49.183048 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d04b1221-b884-4816-9304-39defdddf814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:49.183479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d04b1221-b884-4816-9304-39defdddf814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:49.184175 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d04b1221-b884-4816-9304-39defdddf814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:49.184627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d04b1221-b884-4816-9304-39defdddf814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:49.185048 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d04b1221-b884-4816-9304-39defdddf814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:49.215817 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d04b1221-b884-4816-9304-39defdddf814 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046676 Sep 08 14:02:49.216402 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 663/1433] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:49 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:49.697136 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f867ed4a-8030-4a72-b403-7ab6450ba352 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:49.699339 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f867ed4a-8030-4a72-b403-7ab6450ba352 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:49.705779 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f867ed4a-8030-4a72-b403-7ab6450ba352 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:49.706302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f867ed4a-8030-4a72-b403-7ab6450ba352 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:49.706741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f867ed4a-8030-4a72-b403-7ab6450ba352 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:49.707441 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f867ed4a-8030-4a72-b403-7ab6450ba352 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:49.708398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f867ed4a-8030-4a72-b403-7ab6450ba352 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:49.708934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f867ed4a-8030-4a72-b403-7ab6450ba352 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:49.739947 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f867ed4a-8030-4a72-b403-7ab6450ba352 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046710 Sep 08 14:02:49.740506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 771/1434] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:49 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:49.979253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ccbef5dc-93a8-4cc4-a913-fcda1deec029 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:49.979865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ccbef5dc-93a8-4cc4-a913-fcda1deec029 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:49.986480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccbef5dc-93a8-4cc4-a913-fcda1deec029 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:49.986971 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccbef5dc-93a8-4cc4-a913-fcda1deec029 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:49.987405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccbef5dc-93a8-4cc4-a913-fcda1deec029 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:49.988102 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccbef5dc-93a8-4cc4-a913-fcda1deec029 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:49.988551 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccbef5dc-93a8-4cc4-a913-fcda1deec029 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:49.989047 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccbef5dc-93a8-4cc4-a913-fcda1deec029 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:50.018834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0458c196-8c80-49fa-984e-f5063e9729f0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:50.021786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0458c196-8c80-49fa-984e-f5063e9729f0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:50.028200 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0458c196-8c80-49fa-984e-f5063e9729f0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:50.028690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0458c196-8c80-49fa-984e-f5063e9729f0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:50.029129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0458c196-8c80-49fa-984e-f5063e9729f0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:50.029832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0458c196-8c80-49fa-984e-f5063e9729f0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:50.030319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0458c196-8c80-49fa-984e-f5063e9729f0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:50.030747 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0458c196-8c80-49fa-984e-f5063e9729f0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:50.053403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ccbef5dc-93a8-4cc4-a913-fcda1deec029 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:50.061228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ccbef5dc-93a8-4cc4-a913-fcda1deec029 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.086048 Sep 08 14:02:50.061805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 664/1435] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:49 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:50.062874 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0458c196-8c80-49fa-984e-f5063e9729f0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048514 Sep 08 14:02:50.063514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 772/1436] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:50 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:50.114970 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-23a6686a-872a-4eb9-a78e-deec9351bf87 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:50.115501 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-23a6686a-872a-4eb9-a78e-deec9351bf87 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:50.121863 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-23a6686a-872a-4eb9-a78e-deec9351bf87 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:50.122399 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-23a6686a-872a-4eb9-a78e-deec9351bf87 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:50.122844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-23a6686a-872a-4eb9-a78e-deec9351bf87 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:50.123939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-23a6686a-872a-4eb9-a78e-deec9351bf87 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:50.126392 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-23a6686a-872a-4eb9-a78e-deec9351bf87 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:50.126829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-23a6686a-872a-4eb9-a78e-deec9351bf87 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:50.230417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9d4959a5-60a2-46b7-85ff-180a370b00d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:50.232956 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9d4959a5-60a2-46b7-85ff-180a370b00d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:50.239908 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9d4959a5-60a2-46b7-85ff-180a370b00d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:50.240405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9d4959a5-60a2-46b7-85ff-180a370b00d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:50.240863 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9d4959a5-60a2-46b7-85ff-180a370b00d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:50.241601 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9d4959a5-60a2-46b7-85ff-180a370b00d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:50.242057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9d4959a5-60a2-46b7-85ff-180a370b00d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:50.242515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9d4959a5-60a2-46b7-85ff-180a370b00d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:50.274094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-23a6686a-872a-4eb9-a78e-deec9351bf87 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:50.274514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9d4959a5-60a2-46b7-85ff-180a370b00d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049603 Sep 08 14:02:50.275083 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 773/1437] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:50 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:02:50.284652 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-23a6686a-872a-4eb9-a78e-deec9351bf87 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.172330 Sep 08 14:02:50.284652 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 665/1438] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:50 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:50.752900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4e6283f7-b6f6-4c8b-8328-4333a5ff28b0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:50.755086 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4e6283f7-b6f6-4c8b-8328-4333a5ff28b0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:50.761743 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4e6283f7-b6f6-4c8b-8328-4333a5ff28b0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:50.762276 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4e6283f7-b6f6-4c8b-8328-4333a5ff28b0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:50.762726 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4e6283f7-b6f6-4c8b-8328-4333a5ff28b0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:50.763437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4e6283f7-b6f6-4c8b-8328-4333a5ff28b0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:50.763897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4e6283f7-b6f6-4c8b-8328-4333a5ff28b0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:50.764325 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4e6283f7-b6f6-4c8b-8328-4333a5ff28b0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:50.800461 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4e6283f7-b6f6-4c8b-8328-4333a5ff28b0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051411 Sep 08 14:02:50.802077 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 774/1439] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:50 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:51.075985 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-229ecc80-0592-487c-bf8a-3bf5b423e61b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:51.078219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-229ecc80-0592-487c-bf8a-3bf5b423e61b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:51.080755 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:51.086179 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-229ecc80-0592-487c-bf8a-3bf5b423e61b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:51.086668 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-229ecc80-0592-487c-bf8a-3bf5b423e61b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:51.087100 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-229ecc80-0592-487c-bf8a-3bf5b423e61b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:51.087475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-247aef13-b753-40a7-9935-20028f4b7dda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:51.087795 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-229ecc80-0592-487c-bf8a-3bf5b423e61b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:51.088119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-247aef13-b753-40a7-9935-20028f4b7dda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:51.088304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-229ecc80-0592-487c-bf8a-3bf5b423e61b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:51.088728 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-229ecc80-0592-487c-bf8a-3bf5b423e61b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:51.094253 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-247aef13-b753-40a7-9935-20028f4b7dda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:51.098353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-247aef13-b753-40a7-9935-20028f4b7dda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:51.098353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-247aef13-b753-40a7-9935-20028f4b7dda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:51.098353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-247aef13-b753-40a7-9935-20028f4b7dda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:51.098353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-247aef13-b753-40a7-9935-20028f4b7dda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:51.099724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-247aef13-b753-40a7-9935-20028f4b7dda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:51.120412 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-229ecc80-0592-487c-bf8a-3bf5b423e61b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048344 Sep 08 14:02:51.121002 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 666/1440] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:51 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:51.163994 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-247aef13-b753-40a7-9935-20028f4b7dda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:51.172719 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-247aef13-b753-40a7-9935-20028f4b7dda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.091173 Sep 08 14:02:51.173295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 775/1441] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:51 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:51.287917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e6d093d2-16fe-4356-a5d1-81b7b74ab928 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:51.290047 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e6d093d2-16fe-4356-a5d1-81b7b74ab928 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:51.296586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6d093d2-16fe-4356-a5d1-81b7b74ab928 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:51.297085 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6d093d2-16fe-4356-a5d1-81b7b74ab928 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:51.297518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6d093d2-16fe-4356-a5d1-81b7b74ab928 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:51.298246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6d093d2-16fe-4356-a5d1-81b7b74ab928 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:51.298703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6d093d2-16fe-4356-a5d1-81b7b74ab928 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:51.299125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6d093d2-16fe-4356-a5d1-81b7b74ab928 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:51.307401 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5f606e2a-2e1c-4138-b3d4-7a49b07bbaf8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:51.308008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5f606e2a-2e1c-4138-b3d4-7a49b07bbaf8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:51.314044 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f606e2a-2e1c-4138-b3d4-7a49b07bbaf8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:51.314603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f606e2a-2e1c-4138-b3d4-7a49b07bbaf8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:51.315061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f606e2a-2e1c-4138-b3d4-7a49b07bbaf8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:51.315749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f606e2a-2e1c-4138-b3d4-7a49b07bbaf8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:51.316204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f606e2a-2e1c-4138-b3d4-7a49b07bbaf8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:51.316629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f606e2a-2e1c-4138-b3d4-7a49b07bbaf8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:51.321360 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:51.333675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e6d093d2-16fe-4356-a5d1-81b7b74ab928 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048390 Sep 08 14:02:51.334289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 667/1442] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:51 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:51.457093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5f606e2a-2e1c-4138-b3d4-7a49b07bbaf8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:51.468610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5f606e2a-2e1c-4138-b3d4-7a49b07bbaf8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.165033 Sep 08 14:02:51.469189 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 776/1443] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:51 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:51.814314 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-56ca387c-53e5-4799-92de-53a65409dc93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:51.816479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-56ca387c-53e5-4799-92de-53a65409dc93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:51.823243 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-56ca387c-53e5-4799-92de-53a65409dc93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:51.823724 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-56ca387c-53e5-4799-92de-53a65409dc93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:51.824153 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-56ca387c-53e5-4799-92de-53a65409dc93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:51.824848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-56ca387c-53e5-4799-92de-53a65409dc93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:51.825293 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-56ca387c-53e5-4799-92de-53a65409dc93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:51.825709 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-56ca387c-53e5-4799-92de-53a65409dc93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:51.856450 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-56ca387c-53e5-4799-92de-53a65409dc93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046109 Sep 08 14:02:51.857024 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 668/1444] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:51 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:52.133336 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-19e8f099-8b42-4bce-9f71-466b39154afd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:52.135527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-19e8f099-8b42-4bce-9f71-466b39154afd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:52.141926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-19e8f099-8b42-4bce-9f71-466b39154afd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:52.142446 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-19e8f099-8b42-4bce-9f71-466b39154afd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:52.142885 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-19e8f099-8b42-4bce-9f71-466b39154afd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:52.143575 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-19e8f099-8b42-4bce-9f71-466b39154afd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:52.144032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-19e8f099-8b42-4bce-9f71-466b39154afd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:52.144462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-19e8f099-8b42-4bce-9f71-466b39154afd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:52.173702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-19e8f099-8b42-4bce-9f71-466b39154afd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044862 Sep 08 14:02:52.174193 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 777/1445] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:52 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:52.195934 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-435abcef-e405-469f-98b7-69c9b7941382 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:52.196514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-435abcef-e405-469f-98b7-69c9b7941382 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:52.203586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-435abcef-e405-469f-98b7-69c9b7941382 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:52.204072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-435abcef-e405-469f-98b7-69c9b7941382 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:52.204505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-435abcef-e405-469f-98b7-69c9b7941382 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:52.205204 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-435abcef-e405-469f-98b7-69c9b7941382 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:52.205658 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-435abcef-e405-469f-98b7-69c9b7941382 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:52.206081 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-435abcef-e405-469f-98b7-69c9b7941382 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:52.272890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-435abcef-e405-469f-98b7-69c9b7941382 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:52.280897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-435abcef-e405-469f-98b7-69c9b7941382 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.089318 Sep 08 14:02:52.281466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 669/1446] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:52 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:52.346322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0e84d3a8-a8df-43d2-8569-0877d5798f85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:52.348977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0e84d3a8-a8df-43d2-8569-0877d5798f85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:52.354976 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e84d3a8-a8df-43d2-8569-0877d5798f85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:52.355460 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e84d3a8-a8df-43d2-8569-0877d5798f85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:52.355887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e84d3a8-a8df-43d2-8569-0877d5798f85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:52.356571 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e84d3a8-a8df-43d2-8569-0877d5798f85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:52.357017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e84d3a8-a8df-43d2-8569-0877d5798f85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:52.357434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e84d3a8-a8df-43d2-8569-0877d5798f85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:52.388275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0e84d3a8-a8df-43d2-8569-0877d5798f85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045949 Sep 08 14:02:52.388826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 778/1447] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:52 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:52.492523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fa92ed58-9c5a-459a-928e-22a931018ea1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:52.493037 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fa92ed58-9c5a-459a-928e-22a931018ea1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:52.499011 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa92ed58-9c5a-459a-928e-22a931018ea1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:52.499482 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa92ed58-9c5a-459a-928e-22a931018ea1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:52.500179 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa92ed58-9c5a-459a-928e-22a931018ea1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:52.500852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa92ed58-9c5a-459a-928e-22a931018ea1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:52.501295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa92ed58-9c5a-459a-928e-22a931018ea1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:52.501711 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa92ed58-9c5a-459a-928e-22a931018ea1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:52.637672 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-fa92ed58-9c5a-459a-928e-22a931018ea1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:52.646757 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fa92ed58-9c5a-459a-928e-22a931018ea1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.158194 Sep 08 14:02:52.647333 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 670/1448] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:52 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:52.869026 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ab581b43-19a3-46f6-92c7-084785539e01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:52.871295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ab581b43-19a3-46f6-92c7-084785539e01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:52.877561 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab581b43-19a3-46f6-92c7-084785539e01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:52.878046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab581b43-19a3-46f6-92c7-084785539e01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:52.878551 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab581b43-19a3-46f6-92c7-084785539e01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:52.879778 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab581b43-19a3-46f6-92c7-084785539e01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:52.880236 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab581b43-19a3-46f6-92c7-084785539e01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:52.880658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab581b43-19a3-46f6-92c7-084785539e01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:52.911569 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ab581b43-19a3-46f6-92c7-084785539e01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046510 Sep 08 14:02:52.912122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 779/1449] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:52 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:53.185584 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-331ca3b4-f4d4-40ef-9d47-7bcf95a7e29c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:53.187886 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-331ca3b4-f4d4-40ef-9d47-7bcf95a7e29c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:53.194664 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-331ca3b4-f4d4-40ef-9d47-7bcf95a7e29c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:53.195154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-331ca3b4-f4d4-40ef-9d47-7bcf95a7e29c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:53.195593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-331ca3b4-f4d4-40ef-9d47-7bcf95a7e29c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:53.196311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-331ca3b4-f4d4-40ef-9d47-7bcf95a7e29c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:53.196856 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-331ca3b4-f4d4-40ef-9d47-7bcf95a7e29c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:53.197188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-331ca3b4-f4d4-40ef-9d47-7bcf95a7e29c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:53.228554 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-331ca3b4-f4d4-40ef-9d47-7bcf95a7e29c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046963 Sep 08 14:02:53.229132 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 671/1450] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:53 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:53.304880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-04bf229d-2e35-406e-905e-64e3adce2deb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:53.305587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-04bf229d-2e35-406e-905e-64e3adce2deb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:53.311868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04bf229d-2e35-406e-905e-64e3adce2deb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:53.312346 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04bf229d-2e35-406e-905e-64e3adce2deb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:53.312781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04bf229d-2e35-406e-905e-64e3adce2deb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:53.313458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04bf229d-2e35-406e-905e-64e3adce2deb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:53.313905 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04bf229d-2e35-406e-905e-64e3adce2deb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:53.314355 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04bf229d-2e35-406e-905e-64e3adce2deb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:53.376857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-04bf229d-2e35-406e-905e-64e3adce2deb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:53.386295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-04bf229d-2e35-406e-905e-64e3adce2deb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.085763 Sep 08 14:02:53.386857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 780/1451] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:53 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:53.401071 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e2487add-b65a-498e-8079-c165fdcef4e0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:53.403294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e2487add-b65a-498e-8079-c165fdcef4e0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:53.409386 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e2487add-b65a-498e-8079-c165fdcef4e0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:53.409857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e2487add-b65a-498e-8079-c165fdcef4e0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:53.410360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e2487add-b65a-498e-8079-c165fdcef4e0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:53.411124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e2487add-b65a-498e-8079-c165fdcef4e0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:53.411565 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e2487add-b65a-498e-8079-c165fdcef4e0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:53.411976 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e2487add-b65a-498e-8079-c165fdcef4e0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:53.415636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:53.441280 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e2487add-b65a-498e-8079-c165fdcef4e0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044279 Sep 08 14:02:53.442466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 672/1452] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:53 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:53.670570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5410f9ef-e1b1-414c-be73-cb7e74946341 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:53.671216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5410f9ef-e1b1-414c-be73-cb7e74946341 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:53.678168 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5410f9ef-e1b1-414c-be73-cb7e74946341 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:53.678658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5410f9ef-e1b1-414c-be73-cb7e74946341 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:53.679100 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5410f9ef-e1b1-414c-be73-cb7e74946341 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:53.679810 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5410f9ef-e1b1-414c-be73-cb7e74946341 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:53.680272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5410f9ef-e1b1-414c-be73-cb7e74946341 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:53.680701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5410f9ef-e1b1-414c-be73-cb7e74946341 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:53.821100 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5410f9ef-e1b1-414c-be73-cb7e74946341 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:53.828916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5410f9ef-e1b1-414c-be73-cb7e74946341 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.162334 Sep 08 14:02:53.829484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 781/1453] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:53 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:53.924627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-160f20e7-dd68-4165-a74d-0ed774425bc8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:53.927489 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-160f20e7-dd68-4165-a74d-0ed774425bc8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:53.934068 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-160f20e7-dd68-4165-a74d-0ed774425bc8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:53.934588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-160f20e7-dd68-4165-a74d-0ed774425bc8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:53.935028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-160f20e7-dd68-4165-a74d-0ed774425bc8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:53.935733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-160f20e7-dd68-4165-a74d-0ed774425bc8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:53.936194 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-160f20e7-dd68-4165-a74d-0ed774425bc8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:53.936619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-160f20e7-dd68-4165-a74d-0ed774425bc8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:53.968073 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-160f20e7-dd68-4165-a74d-0ed774425bc8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047719 Sep 08 14:02:53.968847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 673/1454] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:53 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:54.241217 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b7660ad1-351f-4167-bf80-ca308098df6e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:54.243418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b7660ad1-351f-4167-bf80-ca308098df6e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:54.249335 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7660ad1-351f-4167-bf80-ca308098df6e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:54.250077 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7660ad1-351f-4167-bf80-ca308098df6e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:54.250633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7660ad1-351f-4167-bf80-ca308098df6e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:54.251316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7660ad1-351f-4167-bf80-ca308098df6e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:54.251771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7660ad1-351f-4167-bf80-ca308098df6e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:54.252190 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7660ad1-351f-4167-bf80-ca308098df6e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:54.287450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b7660ad1-351f-4167-bf80-ca308098df6e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050170 Sep 08 14:02:54.288038 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 782/1455] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:54 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:54.409061 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f331e1cf-f83b-495e-be0c-dea05b34e433 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:54.409664 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f331e1cf-f83b-495e-be0c-dea05b34e433 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:54.416139 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f331e1cf-f83b-495e-be0c-dea05b34e433 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:54.416615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f331e1cf-f83b-495e-be0c-dea05b34e433 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:54.417047 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f331e1cf-f83b-495e-be0c-dea05b34e433 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:54.417738 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f331e1cf-f83b-495e-be0c-dea05b34e433 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:54.418208 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f331e1cf-f83b-495e-be0c-dea05b34e433 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:54.418632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f331e1cf-f83b-495e-be0c-dea05b34e433 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:54.454454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-da4b52d1-a4f6-40c1-bbf1-95aa6e990815 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:54.456611 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-da4b52d1-a4f6-40c1-bbf1-95aa6e990815 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:54.464436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-da4b52d1-a4f6-40c1-bbf1-95aa6e990815 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:54.464916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-da4b52d1-a4f6-40c1-bbf1-95aa6e990815 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:54.465357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-da4b52d1-a4f6-40c1-bbf1-95aa6e990815 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:54.466068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-da4b52d1-a4f6-40c1-bbf1-95aa6e990815 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:54.466558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-da4b52d1-a4f6-40c1-bbf1-95aa6e990815 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:54.466984 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-da4b52d1-a4f6-40c1-bbf1-95aa6e990815 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:54.483909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f331e1cf-f83b-495e-be0c-dea05b34e433 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:54.492078 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f331e1cf-f83b-495e-be0c-dea05b34e433 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.086864 Sep 08 14:02:54.492582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 674/1456] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:54 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:54.499208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-da4b52d1-a4f6-40c1-bbf1-95aa6e990815 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048740 Sep 08 14:02:54.500036 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 783/1457] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:54 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:02:54.856483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9482866d-db7a-42a1-843a-eab78a00efc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:54.857089 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9482866d-db7a-42a1-843a-eab78a00efc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:54.863354 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9482866d-db7a-42a1-843a-eab78a00efc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:54.864169 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9482866d-db7a-42a1-843a-eab78a00efc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:54.864601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9482866d-db7a-42a1-843a-eab78a00efc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:54.865302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9482866d-db7a-42a1-843a-eab78a00efc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:54.865747 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9482866d-db7a-42a1-843a-eab78a00efc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:54.866186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9482866d-db7a-42a1-843a-eab78a00efc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:54.981266 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6c6a0f5e-82b2-4de9-bbcd-736dabe0397d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:54.984691 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6c6a0f5e-82b2-4de9-bbcd-736dabe0397d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:54.994204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c6a0f5e-82b2-4de9-bbcd-736dabe0397d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:54.995037 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c6a0f5e-82b2-4de9-bbcd-736dabe0397d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:54.995581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c6a0f5e-82b2-4de9-bbcd-736dabe0397d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:54.996469 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c6a0f5e-82b2-4de9-bbcd-736dabe0397d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:54.997071 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c6a0f5e-82b2-4de9-bbcd-736dabe0397d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:54.997503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c6a0f5e-82b2-4de9-bbcd-736dabe0397d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:55.000063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9482866d-db7a-42a1-843a-eab78a00efc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:55.008379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9482866d-db7a-42a1-843a-eab78a00efc2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.160175 Sep 08 14:02:55.008958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 675/1458] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:54 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:55.037585 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6c6a0f5e-82b2-4de9-bbcd-736dabe0397d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060505 Sep 08 14:02:55.038191 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 784/1459] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:54 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:55.300292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0da5c96e-138c-4dcf-90a2-e572f218f8c8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:55.303037 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0da5c96e-138c-4dcf-90a2-e572f218f8c8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:55.309122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0da5c96e-138c-4dcf-90a2-e572f218f8c8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:55.309601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0da5c96e-138c-4dcf-90a2-e572f218f8c8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:55.310033 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0da5c96e-138c-4dcf-90a2-e572f218f8c8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:55.310752 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0da5c96e-138c-4dcf-90a2-e572f218f8c8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:55.311201 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0da5c96e-138c-4dcf-90a2-e572f218f8c8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:55.311618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0da5c96e-138c-4dcf-90a2-e572f218f8c8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:55.341040 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0da5c96e-138c-4dcf-90a2-e572f218f8c8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044805 Sep 08 14:02:55.341614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 676/1460] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:55 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:55.512428 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d052bb9a-0422-46d6-b051-13cef67019d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:55.514606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d052bb9a-0422-46d6-b051-13cef67019d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:55.515392 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d021e684-890b-4e8f-83f6-45f89e58e466 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:55.515999 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d021e684-890b-4e8f-83f6-45f89e58e466 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:55.521381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d052bb9a-0422-46d6-b051-13cef67019d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:55.521898 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d052bb9a-0422-46d6-b051-13cef67019d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:55.522349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d052bb9a-0422-46d6-b051-13cef67019d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:55.523047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d052bb9a-0422-46d6-b051-13cef67019d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:55.523499 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d052bb9a-0422-46d6-b051-13cef67019d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:55.524198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d021e684-890b-4e8f-83f6-45f89e58e466 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:55.524331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d052bb9a-0422-46d6-b051-13cef67019d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:55.524419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d021e684-890b-4e8f-83f6-45f89e58e466 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:55.524755 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d021e684-890b-4e8f-83f6-45f89e58e466 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:55.525461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d021e684-890b-4e8f-83f6-45f89e58e466 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:55.525913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d021e684-890b-4e8f-83f6-45f89e58e466 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:55.526388 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d021e684-890b-4e8f-83f6-45f89e58e466 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:55.556358 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d052bb9a-0422-46d6-b051-13cef67019d1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047838 Sep 08 14:02:55.556849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 785/1461] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:55 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:55.594049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d021e684-890b-4e8f-83f6-45f89e58e466 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:55.606427 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d021e684-890b-4e8f-83f6-45f89e58e466 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.094468 Sep 08 14:02:55.607375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 677/1462] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:55 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:02:56.032572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-30bf9168-e20d-4699-87cf-f75570faaa83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:56.033178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-30bf9168-e20d-4699-87cf-f75570faaa83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:56.040128 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-30bf9168-e20d-4699-87cf-f75570faaa83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:56.040650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-30bf9168-e20d-4699-87cf-f75570faaa83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:56.041093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-30bf9168-e20d-4699-87cf-f75570faaa83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:56.041831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-30bf9168-e20d-4699-87cf-f75570faaa83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:56.042306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-30bf9168-e20d-4699-87cf-f75570faaa83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:56.042730 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-30bf9168-e20d-4699-87cf-f75570faaa83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:56.050119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-97f2c9cb-b358-4f50-90a8-0fb703a38b9e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:56.052388 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-97f2c9cb-b358-4f50-90a8-0fb703a38b9e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:56.058660 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-97f2c9cb-b358-4f50-90a8-0fb703a38b9e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:56.059152 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-97f2c9cb-b358-4f50-90a8-0fb703a38b9e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:56.059905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-97f2c9cb-b358-4f50-90a8-0fb703a38b9e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:56.060294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-97f2c9cb-b358-4f50-90a8-0fb703a38b9e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:56.060770 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-97f2c9cb-b358-4f50-90a8-0fb703a38b9e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:56.061174 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-97f2c9cb-b358-4f50-90a8-0fb703a38b9e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:56.100683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-97f2c9cb-b358-4f50-90a8-0fb703a38b9e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054132 Sep 08 14:02:56.101485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 678/1463] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:56 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:56.189321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-30bf9168-e20d-4699-87cf-f75570faaa83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:56.198276 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-30bf9168-e20d-4699-87cf-f75570faaa83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.169432 Sep 08 14:02:56.198771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 786/1464] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:56 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:56.353210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9107fb09-40a5-41ff-a92c-3c85379ccc76 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:56.355403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9107fb09-40a5-41ff-a92c-3c85379ccc76 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:56.361354 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9107fb09-40a5-41ff-a92c-3c85379ccc76 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:56.361832 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9107fb09-40a5-41ff-a92c-3c85379ccc76 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:56.362454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9107fb09-40a5-41ff-a92c-3c85379ccc76 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:56.363677 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9107fb09-40a5-41ff-a92c-3c85379ccc76 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:56.364119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9107fb09-40a5-41ff-a92c-3c85379ccc76 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:56.364533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9107fb09-40a5-41ff-a92c-3c85379ccc76 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:56.400115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9107fb09-40a5-41ff-a92c-3c85379ccc76 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050841 Sep 08 14:02:56.400692 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 679/1465] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:56 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:56.571707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-db6bba1c-267e-4f28-9bbf-6d82a38684e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:56.573975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-db6bba1c-267e-4f28-9bbf-6d82a38684e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:56.583051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db6bba1c-267e-4f28-9bbf-6d82a38684e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:56.583817 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db6bba1c-267e-4f28-9bbf-6d82a38684e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:56.584504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db6bba1c-267e-4f28-9bbf-6d82a38684e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:56.585617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db6bba1c-267e-4f28-9bbf-6d82a38684e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:56.586404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db6bba1c-267e-4f28-9bbf-6d82a38684e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:56.587100 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db6bba1c-267e-4f28-9bbf-6d82a38684e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:56.625281 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-db6bba1c-267e-4f28-9bbf-6d82a38684e8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.059002 Sep 08 14:02:56.626115 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 787/1466] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:56 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:56.632941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4d51fbd2-cf96-40a1-91d5-d3e398fb4c2c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:56.633548 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4d51fbd2-cf96-40a1-91d5-d3e398fb4c2c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:56.639876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d51fbd2-cf96-40a1-91d5-d3e398fb4c2c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:56.640353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d51fbd2-cf96-40a1-91d5-d3e398fb4c2c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:56.640776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d51fbd2-cf96-40a1-91d5-d3e398fb4c2c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:56.641600 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d51fbd2-cf96-40a1-91d5-d3e398fb4c2c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:56.642028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d51fbd2-cf96-40a1-91d5-d3e398fb4c2c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:56.642475 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d51fbd2-cf96-40a1-91d5-d3e398fb4c2c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:56.710330 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4d51fbd2-cf96-40a1-91d5-d3e398fb4c2c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:56.721178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4d51fbd2-cf96-40a1-91d5-d3e398fb4c2c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.093601 Sep 08 14:02:56.723031 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 680/1467] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:56 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:57.113709 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5b003072-b2bc-4c9d-92bd-2650ca05c016 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:57.115877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5b003072-b2bc-4c9d-92bd-2650ca05c016 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:57.122405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b003072-b2bc-4c9d-92bd-2650ca05c016 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:57.122889 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b003072-b2bc-4c9d-92bd-2650ca05c016 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:57.123324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b003072-b2bc-4c9d-92bd-2650ca05c016 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:57.124010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b003072-b2bc-4c9d-92bd-2650ca05c016 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:57.124728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b003072-b2bc-4c9d-92bd-2650ca05c016 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:57.125154 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b003072-b2bc-4c9d-92bd-2650ca05c016 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:57.156206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5b003072-b2bc-4c9d-92bd-2650ca05c016 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046394 Sep 08 14:02:57.156787 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 788/1468] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:57 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:57.223397 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6669374f-7694-483d-934e-db605a24155d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:57.224037 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6669374f-7694-483d-934e-db605a24155d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:57.230039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6669374f-7694-483d-934e-db605a24155d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:57.230552 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6669374f-7694-483d-934e-db605a24155d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:57.230987 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6669374f-7694-483d-934e-db605a24155d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:57.231678 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6669374f-7694-483d-934e-db605a24155d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:57.232126 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6669374f-7694-483d-934e-db605a24155d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:57.232546 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6669374f-7694-483d-934e-db605a24155d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:57.367941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6669374f-7694-483d-934e-db605a24155d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:57.376483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6669374f-7694-483d-934e-db605a24155d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.157537 Sep 08 14:02:57.377063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 681/1469] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:57 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:57.411995 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-68249d41-0a17-4c56-a568-372e978927fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:57.414128 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-68249d41-0a17-4c56-a568-372e978927fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:57.420600 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68249d41-0a17-4c56-a568-372e978927fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:57.421079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68249d41-0a17-4c56-a568-372e978927fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:57.421515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68249d41-0a17-4c56-a568-372e978927fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:57.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68249d41-0a17-4c56-a568-372e978927fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:57.422699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68249d41-0a17-4c56-a568-372e978927fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:57.423117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68249d41-0a17-4c56-a568-372e978927fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:57.452743 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-68249d41-0a17-4c56-a568-372e978927fd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044733 Sep 08 14:02:57.453298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 789/1470] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:57 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:57.638070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-36bcdd4f-22d0-4fad-bc90-49b1b12297a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:57.640308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-36bcdd4f-22d0-4fad-bc90-49b1b12297a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:57.647750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-36bcdd4f-22d0-4fad-bc90-49b1b12297a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:57.648246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-36bcdd4f-22d0-4fad-bc90-49b1b12297a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:57.648686 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-36bcdd4f-22d0-4fad-bc90-49b1b12297a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:57.649395 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-36bcdd4f-22d0-4fad-bc90-49b1b12297a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:57.649851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-36bcdd4f-22d0-4fad-bc90-49b1b12297a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:57.650310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-36bcdd4f-22d0-4fad-bc90-49b1b12297a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:57.680262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-36bcdd4f-22d0-4fad-bc90-49b1b12297a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046438 Sep 08 14:02:57.680829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 682/1471] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:57 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:57.745136 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-aefdf1ba-37ac-4d79-9d3c-dd8e4532eb47 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:57.745748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-aefdf1ba-37ac-4d79-9d3c-dd8e4532eb47 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:57.747690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:57.752622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aefdf1ba-37ac-4d79-9d3c-dd8e4532eb47 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:57.753133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aefdf1ba-37ac-4d79-9d3c-dd8e4532eb47 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:57.753695 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aefdf1ba-37ac-4d79-9d3c-dd8e4532eb47 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:57.754987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aefdf1ba-37ac-4d79-9d3c-dd8e4532eb47 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:57.755449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aefdf1ba-37ac-4d79-9d3c-dd8e4532eb47 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:57.755881 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aefdf1ba-37ac-4d79-9d3c-dd8e4532eb47 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:57.818627 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-aefdf1ba-37ac-4d79-9d3c-dd8e4532eb47 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:57.826409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-aefdf1ba-37ac-4d79-9d3c-dd8e4532eb47 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.085180 Sep 08 14:02:57.826979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 790/1472] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:57 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:58.170614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-22305d71-6103-45ec-87f5-215b3f961a75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:58.173005 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-22305d71-6103-45ec-87f5-215b3f961a75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:58.179681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-22305d71-6103-45ec-87f5-215b3f961a75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:58.180182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-22305d71-6103-45ec-87f5-215b3f961a75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:58.180589 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-22305d71-6103-45ec-87f5-215b3f961a75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:58.181273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-22305d71-6103-45ec-87f5-215b3f961a75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:58.182174 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-22305d71-6103-45ec-87f5-215b3f961a75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:58.182377 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-22305d71-6103-45ec-87f5-215b3f961a75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:58.212833 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-22305d71-6103-45ec-87f5-215b3f961a75 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047039 Sep 08 14:02:58.213398 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 683/1473] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:58 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:58.401088 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ba353759-629a-469c-8a7a-fa5c474aba2e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:58.401690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ba353759-629a-469c-8a7a-fa5c474aba2e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:58.409425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba353759-629a-469c-8a7a-fa5c474aba2e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:58.410572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba353759-629a-469c-8a7a-fa5c474aba2e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:58.411183 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba353759-629a-469c-8a7a-fa5c474aba2e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:58.411942 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba353759-629a-469c-8a7a-fa5c474aba2e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:58.412404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba353759-629a-469c-8a7a-fa5c474aba2e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:58.413029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba353759-629a-469c-8a7a-fa5c474aba2e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:58.465610 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0e28c0cf-c0d4-4691-b2c4-5172b3214d33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:58.467822 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0e28c0cf-c0d4-4691-b2c4-5172b3214d33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:58.476058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e28c0cf-c0d4-4691-b2c4-5172b3214d33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:58.476550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e28c0cf-c0d4-4691-b2c4-5172b3214d33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:58.476976 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e28c0cf-c0d4-4691-b2c4-5172b3214d33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:58.477681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e28c0cf-c0d4-4691-b2c4-5172b3214d33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:58.478159 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e28c0cf-c0d4-4691-b2c4-5172b3214d33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:58.478569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e28c0cf-c0d4-4691-b2c4-5172b3214d33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:58.510609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0e28c0cf-c0d4-4691-b2c4-5172b3214d33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048944 Sep 08 14:02:58.511196 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 684/1474] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:58 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:02:58.563126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ba353759-629a-469c-8a7a-fa5c474aba2e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:58.571466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ba353759-629a-469c-8a7a-fa5c474aba2e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.174628 Sep 08 14:02:58.572031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 791/1475] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:58 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:58.693137 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-18e0794c-ef2a-43d3-b85f-be1b96688e60 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:58.695374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-18e0794c-ef2a-43d3-b85f-be1b96688e60 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:58.701833 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18e0794c-ef2a-43d3-b85f-be1b96688e60 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:58.702358 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18e0794c-ef2a-43d3-b85f-be1b96688e60 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:58.702803 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18e0794c-ef2a-43d3-b85f-be1b96688e60 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:58.703496 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18e0794c-ef2a-43d3-b85f-be1b96688e60 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:58.703953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18e0794c-ef2a-43d3-b85f-be1b96688e60 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:58.704374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18e0794c-ef2a-43d3-b85f-be1b96688e60 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:58.735518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-18e0794c-ef2a-43d3-b85f-be1b96688e60 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046407 Sep 08 14:02:58.736002 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 685/1476] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:58 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:58.849224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-69bc68ae-dd02-49af-b997-bfdec51a1a95 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:58.849905 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-69bc68ae-dd02-49af-b997-bfdec51a1a95 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:58.856073 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69bc68ae-dd02-49af-b997-bfdec51a1a95 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:58.856556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69bc68ae-dd02-49af-b997-bfdec51a1a95 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:58.856989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69bc68ae-dd02-49af-b997-bfdec51a1a95 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:58.857675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69bc68ae-dd02-49af-b997-bfdec51a1a95 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:58.858123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69bc68ae-dd02-49af-b997-bfdec51a1a95 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:58.858575 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69bc68ae-dd02-49af-b997-bfdec51a1a95 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:58.923442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-69bc68ae-dd02-49af-b997-bfdec51a1a95 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:58.925631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:58.931922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-69bc68ae-dd02-49af-b997-bfdec51a1a95 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.086590 Sep 08 14:02:58.932488 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 792/1477] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:58 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:59.224994 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-85b4acfe-2eff-42af-9ca0-73b374b2a8d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:59.227149 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-85b4acfe-2eff-42af-9ca0-73b374b2a8d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:59.233199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85b4acfe-2eff-42af-9ca0-73b374b2a8d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:59.233679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85b4acfe-2eff-42af-9ca0-73b374b2a8d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:59.234109 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85b4acfe-2eff-42af-9ca0-73b374b2a8d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:59.234814 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85b4acfe-2eff-42af-9ca0-73b374b2a8d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:59.235260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85b4acfe-2eff-42af-9ca0-73b374b2a8d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:59.235681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85b4acfe-2eff-42af-9ca0-73b374b2a8d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:59.265827 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-85b4acfe-2eff-42af-9ca0-73b374b2a8d3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044818 Sep 08 14:02:59.266410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 686/1478] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:59 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:59.524186 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-dfd06a3f-76f9-4eee-ba45-27880b49f793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:59.526440 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-dfd06a3f-76f9-4eee-ba45-27880b49f793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:59.533470 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dfd06a3f-76f9-4eee-ba45-27880b49f793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:59.533915 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dfd06a3f-76f9-4eee-ba45-27880b49f793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:59.534385 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dfd06a3f-76f9-4eee-ba45-27880b49f793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:59.535787 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dfd06a3f-76f9-4eee-ba45-27880b49f793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:59.536241 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dfd06a3f-76f9-4eee-ba45-27880b49f793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:59.536664 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dfd06a3f-76f9-4eee-ba45-27880b49f793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:59.567441 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-dfd06a3f-76f9-4eee-ba45-27880b49f793 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047407 Sep 08 14:02:59.567995 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 793/1479] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:02:59 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:02:59.595088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6024ead7-87d1-4cb2-a657-04c4f6f4e7cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:59.595694 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6024ead7-87d1-4cb2-a657-04c4f6f4e7cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:59.601700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6024ead7-87d1-4cb2-a657-04c4f6f4e7cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:59.602208 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6024ead7-87d1-4cb2-a657-04c4f6f4e7cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:59.602644 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6024ead7-87d1-4cb2-a657-04c4f6f4e7cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:59.603324 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6024ead7-87d1-4cb2-a657-04c4f6f4e7cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:59.603773 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6024ead7-87d1-4cb2-a657-04c4f6f4e7cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:59.604191 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6024ead7-87d1-4cb2-a657-04c4f6f4e7cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:59.738286 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6024ead7-87d1-4cb2-a657-04c4f6f4e7cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:02:59.747027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6024ead7-87d1-4cb2-a657-04c4f6f4e7cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.155294 Sep 08 14:02:59.747172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 687/1480] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:59 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:02:59.747629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-425ff0a5-c194-42cb-8fce-a394758376f0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:59.750231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-425ff0a5-c194-42cb-8fce-a394758376f0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:59.757124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-425ff0a5-c194-42cb-8fce-a394758376f0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:59.757685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-425ff0a5-c194-42cb-8fce-a394758376f0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:59.758153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-425ff0a5-c194-42cb-8fce-a394758376f0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:59.758855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-425ff0a5-c194-42cb-8fce-a394758376f0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:59.759309 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-425ff0a5-c194-42cb-8fce-a394758376f0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:59.759735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-425ff0a5-c194-42cb-8fce-a394758376f0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:59.790280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-425ff0a5-c194-42cb-8fce-a394758376f0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046532 Sep 08 14:02:59.790760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 794/1481] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:02:59 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:02:59.963531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3d318cd1-eb54-4c8c-88cc-6ebab50e0434 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:02:59.964134 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3d318cd1-eb54-4c8c-88cc-6ebab50e0434 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:02:59.970067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d318cd1-eb54-4c8c-88cc-6ebab50e0434 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:59.970576 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d318cd1-eb54-4c8c-88cc-6ebab50e0434 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:59.971008 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d318cd1-eb54-4c8c-88cc-6ebab50e0434 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:02:59.971690 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d318cd1-eb54-4c8c-88cc-6ebab50e0434 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:02:59.972198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d318cd1-eb54-4c8c-88cc-6ebab50e0434 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:02:59.972622 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d318cd1-eb54-4c8c-88cc-6ebab50e0434 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:00.034342 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3d318cd1-eb54-4c8c-88cc-6ebab50e0434 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:00.037808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:00.042471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3d318cd1-eb54-4c8c-88cc-6ebab50e0434 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.082778 Sep 08 14:03:00.042981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 688/1482] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:02:59 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:00.279205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1857f714-aafc-4416-877c-1a7a2cd46b7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:00.281426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1857f714-aafc-4416-877c-1a7a2cd46b7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:00.287910 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1857f714-aafc-4416-877c-1a7a2cd46b7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:00.288404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1857f714-aafc-4416-877c-1a7a2cd46b7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:00.288850 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1857f714-aafc-4416-877c-1a7a2cd46b7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:00.289549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1857f714-aafc-4416-877c-1a7a2cd46b7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:00.290010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1857f714-aafc-4416-877c-1a7a2cd46b7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:00.290543 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1857f714-aafc-4416-877c-1a7a2cd46b7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:00.320219 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1857f714-aafc-4416-877c-1a7a2cd46b7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044980 Sep 08 14:03:00.320701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 795/1483] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:00 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:00.579393 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4200978c-da5b-46c8-83de-88837d5fb752 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:00.581530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4200978c-da5b-46c8-83de-88837d5fb752 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:00.588036 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4200978c-da5b-46c8-83de-88837d5fb752 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:00.588550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4200978c-da5b-46c8-83de-88837d5fb752 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:00.588988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4200978c-da5b-46c8-83de-88837d5fb752 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:00.589708 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4200978c-da5b-46c8-83de-88837d5fb752 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:00.590185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4200978c-da5b-46c8-83de-88837d5fb752 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:00.590614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4200978c-da5b-46c8-83de-88837d5fb752 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:00.620621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4200978c-da5b-46c8-83de-88837d5fb752 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045361 Sep 08 14:03:00.621187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 689/1484] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:00 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:00.771450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a24fd2e9-3a56-4a90-bb9d-28d4ec22f634 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:00.772070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a24fd2e9-3a56-4a90-bb9d-28d4ec22f634 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:00.778676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a24fd2e9-3a56-4a90-bb9d-28d4ec22f634 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:00.779166 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a24fd2e9-3a56-4a90-bb9d-28d4ec22f634 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:00.779610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a24fd2e9-3a56-4a90-bb9d-28d4ec22f634 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:00.780305 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a24fd2e9-3a56-4a90-bb9d-28d4ec22f634 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:00.780768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a24fd2e9-3a56-4a90-bb9d-28d4ec22f634 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:00.781212 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a24fd2e9-3a56-4a90-bb9d-28d4ec22f634 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:00.802907 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1048f201-4a52-44d4-b18d-e707f7ae0c6b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:00.805090 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1048f201-4a52-44d4-b18d-e707f7ae0c6b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:00.811653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1048f201-4a52-44d4-b18d-e707f7ae0c6b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:00.812131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1048f201-4a52-44d4-b18d-e707f7ae0c6b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:00.812570 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1048f201-4a52-44d4-b18d-e707f7ae0c6b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:00.813268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1048f201-4a52-44d4-b18d-e707f7ae0c6b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:00.813721 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1048f201-4a52-44d4-b18d-e707f7ae0c6b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:00.814161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1048f201-4a52-44d4-b18d-e707f7ae0c6b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:00.846236 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1048f201-4a52-44d4-b18d-e707f7ae0c6b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047334 Sep 08 14:03:00.847785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 690/1485] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:00 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:03:00.915497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a24fd2e9-3a56-4a90-bb9d-28d4ec22f634 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:00.923345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a24fd2e9-3a56-4a90-bb9d-28d4ec22f634 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.156348 Sep 08 14:03:00.923910 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 796/1486] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:00 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:01.072754 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2786e256-c2b2-4473-be32-a35503dd32b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:01.073364 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2786e256-c2b2-4473-be32-a35503dd32b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:01.079287 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2786e256-c2b2-4473-be32-a35503dd32b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:01.079767 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2786e256-c2b2-4473-be32-a35503dd32b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:01.080201 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2786e256-c2b2-4473-be32-a35503dd32b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:01.080887 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2786e256-c2b2-4473-be32-a35503dd32b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:01.082054 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2786e256-c2b2-4473-be32-a35503dd32b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:01.082502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2786e256-c2b2-4473-be32-a35503dd32b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:01.142651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-2786e256-c2b2-4473-be32-a35503dd32b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:01.151051 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2786e256-c2b2-4473-be32-a35503dd32b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.082162 Sep 08 14:03:01.151637 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 691/1487] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:01 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:01.333231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-494b5a41-229a-4dfe-81f8-096dff817b12 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:01.335399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-494b5a41-229a-4dfe-81f8-096dff817b12 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:01.341486 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-494b5a41-229a-4dfe-81f8-096dff817b12 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:01.341970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-494b5a41-229a-4dfe-81f8-096dff817b12 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:01.342448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-494b5a41-229a-4dfe-81f8-096dff817b12 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:01.343141 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-494b5a41-229a-4dfe-81f8-096dff817b12 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:01.343592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-494b5a41-229a-4dfe-81f8-096dff817b12 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:01.344017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-494b5a41-229a-4dfe-81f8-096dff817b12 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:01.373384 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-494b5a41-229a-4dfe-81f8-096dff817b12 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044682 Sep 08 14:03:01.373939 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 797/1488] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:01 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:01.634436 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b6c60ac2-bbe6-46f8-b369-0674bb7b2b5b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:01.636812 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b6c60ac2-bbe6-46f8-b369-0674bb7b2b5b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:01.644079 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b6c60ac2-bbe6-46f8-b369-0674bb7b2b5b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:01.644079 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b6c60ac2-bbe6-46f8-b369-0674bb7b2b5b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:01.644451 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b6c60ac2-bbe6-46f8-b369-0674bb7b2b5b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:01.645154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b6c60ac2-bbe6-46f8-b369-0674bb7b2b5b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:01.645599 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b6c60ac2-bbe6-46f8-b369-0674bb7b2b5b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:01.646015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b6c60ac2-bbe6-46f8-b369-0674bb7b2b5b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:01.677569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b6c60ac2-bbe6-46f8-b369-0674bb7b2b5b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046994 Sep 08 14:03:01.678161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 692/1489] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:01 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:01.859996 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-abf61c02-57b8-4d91-b145-ce3d5b379548 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:01.863425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-abf61c02-57b8-4d91-b145-ce3d5b379548 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:01.870685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-abf61c02-57b8-4d91-b145-ce3d5b379548 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:01.871178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-abf61c02-57b8-4d91-b145-ce3d5b379548 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:01.871684 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-abf61c02-57b8-4d91-b145-ce3d5b379548 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:01.872938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-abf61c02-57b8-4d91-b145-ce3d5b379548 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:01.873804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-abf61c02-57b8-4d91-b145-ce3d5b379548 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:01.874904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-abf61c02-57b8-4d91-b145-ce3d5b379548 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:01.909901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-abf61c02-57b8-4d91-b145-ce3d5b379548 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053381 Sep 08 14:03:01.910587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 798/1490] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:01 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:01.951208 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8a81d421-4306-42aa-8488-4719606a8e83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:01.951208 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8a81d421-4306-42aa-8488-4719606a8e83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:01.956492 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8a81d421-4306-42aa-8488-4719606a8e83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:01.957406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8a81d421-4306-42aa-8488-4719606a8e83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:01.957597 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8a81d421-4306-42aa-8488-4719606a8e83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:01.958222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8a81d421-4306-42aa-8488-4719606a8e83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:01.958637 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8a81d421-4306-42aa-8488-4719606a8e83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:01.959056 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8a81d421-4306-42aa-8488-4719606a8e83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:02.101207 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8a81d421-4306-42aa-8488-4719606a8e83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:02.110303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8a81d421-4306-42aa-8488-4719606a8e83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.165657 Sep 08 14:03:02.110303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 693/1491] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:01 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:02.174264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b5cdb645-0c2f-472f-afd4-034d23137ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:02.175549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b5cdb645-0c2f-472f-afd4-034d23137ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:02.181663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5cdb645-0c2f-472f-afd4-034d23137ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:02.182206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5cdb645-0c2f-472f-afd4-034d23137ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:02.182642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5cdb645-0c2f-472f-afd4-034d23137ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:02.183364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5cdb645-0c2f-472f-afd4-034d23137ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:02.183783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5cdb645-0c2f-472f-afd4-034d23137ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:02.184203 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5cdb645-0c2f-472f-afd4-034d23137ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:02.272217 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b5cdb645-0c2f-472f-afd4-034d23137ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:02.285978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b5cdb645-0c2f-472f-afd4-034d23137ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.115842 Sep 08 14:03:02.286634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 799/1492] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:02 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:02.387912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-85d9750e-3043-4a02-99c1-a783d36569b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:02.389938 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-85d9750e-3043-4a02-99c1-a783d36569b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:02.396752 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85d9750e-3043-4a02-99c1-a783d36569b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:02.397225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85d9750e-3043-4a02-99c1-a783d36569b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:02.397662 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85d9750e-3043-4a02-99c1-a783d36569b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:02.398373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85d9750e-3043-4a02-99c1-a783d36569b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:02.398820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85d9750e-3043-4a02-99c1-a783d36569b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:02.399235 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85d9750e-3043-4a02-99c1-a783d36569b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:02.429796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-85d9750e-3043-4a02-99c1-a783d36569b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046292 Sep 08 14:03:02.430388 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 694/1493] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:02 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:02.690861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7aa4cebb-ef34-44ff-9dea-bba6142a6e32 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:02.693361 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7aa4cebb-ef34-44ff-9dea-bba6142a6e32 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:02.701875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7aa4cebb-ef34-44ff-9dea-bba6142a6e32 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:02.702745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7aa4cebb-ef34-44ff-9dea-bba6142a6e32 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:02.702745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7aa4cebb-ef34-44ff-9dea-bba6142a6e32 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:02.703371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7aa4cebb-ef34-44ff-9dea-bba6142a6e32 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:02.703821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7aa4cebb-ef34-44ff-9dea-bba6142a6e32 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:02.704242 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7aa4cebb-ef34-44ff-9dea-bba6142a6e32 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:02.735421 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7aa4cebb-ef34-44ff-9dea-bba6142a6e32 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049225 Sep 08 14:03:02.735982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 800/1494] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:02 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:02.923133 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-568c4b04-ff11-42e2-a538-014768df080b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:02.925514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-568c4b04-ff11-42e2-a538-014768df080b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:02.934572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-568c4b04-ff11-42e2-a538-014768df080b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:02.944848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-568c4b04-ff11-42e2-a538-014768df080b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:02.944848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-568c4b04-ff11-42e2-a538-014768df080b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:02.944848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-568c4b04-ff11-42e2-a538-014768df080b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:02.944848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-568c4b04-ff11-42e2-a538-014768df080b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:02.945438 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-568c4b04-ff11-42e2-a538-014768df080b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:02.983288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-568c4b04-ff11-42e2-a538-014768df080b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.064068 Sep 08 14:03:02.983989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 695/1495] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:02 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:03:03.136400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f72972f7-f70e-4e1d-bd43-7a4e5df12feb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:03.137049 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f72972f7-f70e-4e1d-bd43-7a4e5df12feb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:03.143896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f72972f7-f70e-4e1d-bd43-7a4e5df12feb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:03.144427 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f72972f7-f70e-4e1d-bd43-7a4e5df12feb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:03.144827 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f72972f7-f70e-4e1d-bd43-7a4e5df12feb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:03.145709 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f72972f7-f70e-4e1d-bd43-7a4e5df12feb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:03.146973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f72972f7-f70e-4e1d-bd43-7a4e5df12feb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:03.146973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f72972f7-f70e-4e1d-bd43-7a4e5df12feb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:03.292513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f72972f7-f70e-4e1d-bd43-7a4e5df12feb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:03.307606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f72972f7-f70e-4e1d-bd43-7a4e5df12feb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.177494 Sep 08 14:03:03.308386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 801/1496] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:03 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:03.312611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3afe9284-00c7-4a72-9ca0-8224975a6bbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:03.313685 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3afe9284-00c7-4a72-9ca0-8224975a6bbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:03.324054 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3afe9284-00c7-4a72-9ca0-8224975a6bbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:03.324808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3afe9284-00c7-4a72-9ca0-8224975a6bbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:03.328390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3afe9284-00c7-4a72-9ca0-8224975a6bbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:03.328390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3afe9284-00c7-4a72-9ca0-8224975a6bbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:03.328390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3afe9284-00c7-4a72-9ca0-8224975a6bbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:03.328390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3afe9284-00c7-4a72-9ca0-8224975a6bbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:03.407089 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3afe9284-00c7-4a72-9ca0-8224975a6bbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:03.415856 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3afe9284-00c7-4a72-9ca0-8224975a6bbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.109777 Sep 08 14:03:03.416719 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 696/1497] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:03 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:03.442681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1738838f-2d6a-46a1-ba99-cac09a20fd84 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:03.444792 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1738838f-2d6a-46a1-ba99-cac09a20fd84 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:03.451415 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1738838f-2d6a-46a1-ba99-cac09a20fd84 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:03.451900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1738838f-2d6a-46a1-ba99-cac09a20fd84 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:03.452337 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1738838f-2d6a-46a1-ba99-cac09a20fd84 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:03.453044 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1738838f-2d6a-46a1-ba99-cac09a20fd84 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:03.453503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1738838f-2d6a-46a1-ba99-cac09a20fd84 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:03.453925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1738838f-2d6a-46a1-ba99-cac09a20fd84 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:03.485325 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1738838f-2d6a-46a1-ba99-cac09a20fd84 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046905 Sep 08 14:03:03.485897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 802/1498] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:03 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:03.748516 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cd157109-f5b4-48a9-bfdb-d41a5597e431 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:03.750903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cd157109-f5b4-48a9-bfdb-d41a5597e431 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:03.757705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd157109-f5b4-48a9-bfdb-d41a5597e431 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:03.758232 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd157109-f5b4-48a9-bfdb-d41a5597e431 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:03.758675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd157109-f5b4-48a9-bfdb-d41a5597e431 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:03.759377 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd157109-f5b4-48a9-bfdb-d41a5597e431 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:03.759837 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd157109-f5b4-48a9-bfdb-d41a5597e431 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:03.760262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd157109-f5b4-48a9-bfdb-d41a5597e431 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:03.789232 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cd157109-f5b4-48a9-bfdb-d41a5597e431 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044902 Sep 08 14:03:03.789804 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 697/1499] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:03 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:03.996868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-22813a6a-1ad2-4175-9535-f57eb6a89c2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:03.999393 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-22813a6a-1ad2-4175-9535-f57eb6a89c2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:04.006411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22813a6a-1ad2-4175-9535-f57eb6a89c2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:04.006911 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22813a6a-1ad2-4175-9535-f57eb6a89c2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:04.007356 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22813a6a-1ad2-4175-9535-f57eb6a89c2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:04.008064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22813a6a-1ad2-4175-9535-f57eb6a89c2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:04.008526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22813a6a-1ad2-4175-9535-f57eb6a89c2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:04.008956 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22813a6a-1ad2-4175-9535-f57eb6a89c2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:04.039636 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-22813a6a-1ad2-4175-9535-f57eb6a89c2f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046744 Sep 08 14:03:04.040201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 803/1500] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:03 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:04.330663 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3965db78-5301-4cde-bc73-681e6b8ee3bf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:04.331229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3965db78-5301-4cde-bc73-681e6b8ee3bf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:04.337711 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3965db78-5301-4cde-bc73-681e6b8ee3bf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:04.338219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3965db78-5301-4cde-bc73-681e6b8ee3bf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:04.338653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3965db78-5301-4cde-bc73-681e6b8ee3bf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:04.339345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3965db78-5301-4cde-bc73-681e6b8ee3bf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:04.339791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3965db78-5301-4cde-bc73-681e6b8ee3bf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:04.340209 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3965db78-5301-4cde-bc73-681e6b8ee3bf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:04.442612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2fb0fdca-c82c-4e1c-bb80-86a9193ba645 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:04.443751 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2fb0fdca-c82c-4e1c-bb80-86a9193ba645 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:04.453601 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fb0fdca-c82c-4e1c-bb80-86a9193ba645 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:04.454100 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fb0fdca-c82c-4e1c-bb80-86a9193ba645 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:04.454563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fb0fdca-c82c-4e1c-bb80-86a9193ba645 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:04.455286 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fb0fdca-c82c-4e1c-bb80-86a9193ba645 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:04.456400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fb0fdca-c82c-4e1c-bb80-86a9193ba645 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:04.456839 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fb0fdca-c82c-4e1c-bb80-86a9193ba645 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:04.481711 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3965db78-5301-4cde-bc73-681e6b8ee3bf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:04.493686 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3965db78-5301-4cde-bc73-681e6b8ee3bf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.166955 Sep 08 14:03:04.494312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 698/1501] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:04 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:04.499311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-38e85f57-69f9-45f4-bf44-422352664c17 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:04.501588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-38e85f57-69f9-45f4-bf44-422352664c17 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:04.513106 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38e85f57-69f9-45f4-bf44-422352664c17 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:04.517374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38e85f57-69f9-45f4-bf44-422352664c17 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:04.517374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38e85f57-69f9-45f4-bf44-422352664c17 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:04.517374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38e85f57-69f9-45f4-bf44-422352664c17 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:04.517374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38e85f57-69f9-45f4-bf44-422352664c17 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:04.517844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38e85f57-69f9-45f4-bf44-422352664c17 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:04.524197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2fb0fdca-c82c-4e1c-bb80-86a9193ba645 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:04.532319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2fb0fdca-c82c-4e1c-bb80-86a9193ba645 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.096371 Sep 08 14:03:04.532900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 804/1502] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:04 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:04.548778 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-38e85f57-69f9-45f4-bf44-422352664c17 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053411 Sep 08 14:03:04.549340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 699/1503] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:04 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:04.801972 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-69e92104-a39c-4c92-b10e-d652cd92e8b0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:04.804317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-69e92104-a39c-4c92-b10e-d652cd92e8b0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:04.810491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69e92104-a39c-4c92-b10e-d652cd92e8b0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:04.810986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69e92104-a39c-4c92-b10e-d652cd92e8b0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:04.811425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69e92104-a39c-4c92-b10e-d652cd92e8b0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:04.812132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69e92104-a39c-4c92-b10e-d652cd92e8b0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:04.812593 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69e92104-a39c-4c92-b10e-d652cd92e8b0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:04.813027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-69e92104-a39c-4c92-b10e-d652cd92e8b0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:04.845735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-69e92104-a39c-4c92-b10e-d652cd92e8b0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047717 Sep 08 14:03:04.846379 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 805/1504] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:04 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:05.055086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a8eb750c-6c92-4c31-8ae3-7aeaec7678b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:05.057432 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a8eb750c-6c92-4c31-8ae3-7aeaec7678b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:05.063912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a8eb750c-6c92-4c31-8ae3-7aeaec7678b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:05.064518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a8eb750c-6c92-4c31-8ae3-7aeaec7678b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:05.064894 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a8eb750c-6c92-4c31-8ae3-7aeaec7678b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:05.066938 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a8eb750c-6c92-4c31-8ae3-7aeaec7678b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:05.067426 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a8eb750c-6c92-4c31-8ae3-7aeaec7678b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:05.067860 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a8eb750c-6c92-4c31-8ae3-7aeaec7678b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:05.100815 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a8eb750c-6c92-4c31-8ae3-7aeaec7678b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051738 Sep 08 14:03:05.101394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 700/1505] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:05 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:05.518562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d9f3be82-50ad-45ea-8491-9722f1969d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:05.519159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d9f3be82-50ad-45ea-8491-9722f1969d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:05.525629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d9f3be82-50ad-45ea-8491-9722f1969d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:05.526113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d9f3be82-50ad-45ea-8491-9722f1969d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:05.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d9f3be82-50ad-45ea-8491-9722f1969d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:05.527262 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d9f3be82-50ad-45ea-8491-9722f1969d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:05.527717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d9f3be82-50ad-45ea-8491-9722f1969d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:05.528143 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d9f3be82-50ad-45ea-8491-9722f1969d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:05.555741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3203c7e0-ad63-4829-8cc9-d135db7d6e1a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:05.557010 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3203c7e0-ad63-4829-8cc9-d135db7d6e1a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:05.563454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3203c7e0-ad63-4829-8cc9-d135db7d6e1a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:05.563963 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3203c7e0-ad63-4829-8cc9-d135db7d6e1a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:05.564399 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3203c7e0-ad63-4829-8cc9-d135db7d6e1a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:05.565874 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3203c7e0-ad63-4829-8cc9-d135db7d6e1a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:05.566211 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3203c7e0-ad63-4829-8cc9-d135db7d6e1a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:05.566640 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3203c7e0-ad63-4829-8cc9-d135db7d6e1a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:05.633184 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3203c7e0-ad63-4829-8cc9-d135db7d6e1a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:05.641310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3203c7e0-ad63-4829-8cc9-d135db7d6e1a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.089355 Sep 08 14:03:05.641967 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 701/1506] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:05 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:05.647796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-078e5222-b7fd-425f-8fa0-da604e23dcd0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:05.650060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-078e5222-b7fd-425f-8fa0-da604e23dcd0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:05.655991 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-078e5222-b7fd-425f-8fa0-da604e23dcd0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:05.656552 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-078e5222-b7fd-425f-8fa0-da604e23dcd0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:05.657206 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-078e5222-b7fd-425f-8fa0-da604e23dcd0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:05.657809 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-078e5222-b7fd-425f-8fa0-da604e23dcd0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:05.658310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-078e5222-b7fd-425f-8fa0-da604e23dcd0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:05.659278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-078e5222-b7fd-425f-8fa0-da604e23dcd0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:05.665703 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d9f3be82-50ad-45ea-8491-9722f1969d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:05.673755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d9f3be82-50ad-45ea-8491-9722f1969d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.158918 Sep 08 14:03:05.676678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 806/1507] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:05 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:05.690042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-078e5222-b7fd-425f-8fa0-da604e23dcd0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046915 Sep 08 14:03:05.690641 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 702/1508] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:05 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:05.859105 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5f3f3466-c87a-4e6d-abcc-3a5782cc9f24 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:05.861284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5f3f3466-c87a-4e6d-abcc-3a5782cc9f24 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:05.868022 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f3f3466-c87a-4e6d-abcc-3a5782cc9f24 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:05.868525 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f3f3466-c87a-4e6d-abcc-3a5782cc9f24 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:05.868944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f3f3466-c87a-4e6d-abcc-3a5782cc9f24 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:05.869634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f3f3466-c87a-4e6d-abcc-3a5782cc9f24 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:05.870087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f3f3466-c87a-4e6d-abcc-3a5782cc9f24 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:05.870540 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f3f3466-c87a-4e6d-abcc-3a5782cc9f24 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:05.900481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5f3f3466-c87a-4e6d-abcc-3a5782cc9f24 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045253 Sep 08 14:03:05.901057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 807/1509] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:05 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:06.113614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-956d5d87-aa43-4040-b2d8-8847e5b91b29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:06.115848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-956d5d87-aa43-4040-b2d8-8847e5b91b29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:06.122060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-956d5d87-aa43-4040-b2d8-8847e5b91b29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:06.122574 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-956d5d87-aa43-4040-b2d8-8847e5b91b29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:06.123005 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-956d5d87-aa43-4040-b2d8-8847e5b91b29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:06.124038 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-956d5d87-aa43-4040-b2d8-8847e5b91b29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:06.124485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-956d5d87-aa43-4040-b2d8-8847e5b91b29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:06.124905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-956d5d87-aa43-4040-b2d8-8847e5b91b29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:06.155922 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-956d5d87-aa43-4040-b2d8-8847e5b91b29 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046478 Sep 08 14:03:06.156486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 703/1510] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:06 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:06.664847 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d8836559-d2d9-4ed5-9d23-4a4ca8f8b4b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:06.665448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d8836559-d2d9-4ed5-9d23-4a4ca8f8b4b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:06.671894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d8836559-d2d9-4ed5-9d23-4a4ca8f8b4b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:06.672515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d8836559-d2d9-4ed5-9d23-4a4ca8f8b4b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:06.672956 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d8836559-d2d9-4ed5-9d23-4a4ca8f8b4b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:06.673646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d8836559-d2d9-4ed5-9d23-4a4ca8f8b4b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:06.674101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d8836559-d2d9-4ed5-9d23-4a4ca8f8b4b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:06.674653 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d8836559-d2d9-4ed5-9d23-4a4ca8f8b4b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:06.697306 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-578b8c3d-a4a4-4762-8dcc-3b7ad4fe8dcf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:06.698201 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-578b8c3d-a4a4-4762-8dcc-3b7ad4fe8dcf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:06.704709 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-578b8c3d-a4a4-4762-8dcc-3b7ad4fe8dcf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:06.705197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-578b8c3d-a4a4-4762-8dcc-3b7ad4fe8dcf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:06.705897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-578b8c3d-a4a4-4762-8dcc-3b7ad4fe8dcf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:06.707201 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-578b8c3d-a4a4-4762-8dcc-3b7ad4fe8dcf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:06.708173 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-578b8c3d-a4a4-4762-8dcc-3b7ad4fe8dcf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:06.708609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-578b8c3d-a4a4-4762-8dcc-3b7ad4fe8dcf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:06.737524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d8836559-d2d9-4ed5-9d23-4a4ca8f8b4b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:06.745628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d8836559-d2d9-4ed5-9d23-4a4ca8f8b4b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.084625 Sep 08 14:03:06.746344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 808/1511] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:06 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:06.751436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c60f1c93-fbe9-433c-b996-ae5ae6d95a4b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:06.754093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c60f1c93-fbe9-433c-b996-ae5ae6d95a4b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:06.760372 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c60f1c93-fbe9-433c-b996-ae5ae6d95a4b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:06.760975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c60f1c93-fbe9-433c-b996-ae5ae6d95a4b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:06.761484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c60f1c93-fbe9-433c-b996-ae5ae6d95a4b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:06.762288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c60f1c93-fbe9-433c-b996-ae5ae6d95a4b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:06.762845 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c60f1c93-fbe9-433c-b996-ae5ae6d95a4b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:06.763282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c60f1c93-fbe9-433c-b996-ae5ae6d95a4b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:06.777345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:06.793692 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c60f1c93-fbe9-433c-b996-ae5ae6d95a4b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046216 Sep 08 14:03:06.794275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 809/1512] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:06 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:06.847793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-578b8c3d-a4a4-4762-8dcc-3b7ad4fe8dcf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:06.855734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-578b8c3d-a4a4-4762-8dcc-3b7ad4fe8dcf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.162476 Sep 08 14:03:06.856307 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 704/1513] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:06 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:06.913488 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d4d9ac8c-687e-48a5-a8d0-5c4ced78ed08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:06.915747 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d4d9ac8c-687e-48a5-a8d0-5c4ced78ed08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:06.922373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4d9ac8c-687e-48a5-a8d0-5c4ced78ed08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:06.922863 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4d9ac8c-687e-48a5-a8d0-5c4ced78ed08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:06.923302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4d9ac8c-687e-48a5-a8d0-5c4ced78ed08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:06.923996 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4d9ac8c-687e-48a5-a8d0-5c4ced78ed08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:06.924965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4d9ac8c-687e-48a5-a8d0-5c4ced78ed08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:06.925383 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4d9ac8c-687e-48a5-a8d0-5c4ced78ed08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:06.955713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d4d9ac8c-687e-48a5-a8d0-5c4ced78ed08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046223 Sep 08 14:03:06.956280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 810/1514] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:06 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:07.169171 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e6d5809f-68b8-4938-9012-7c1c43c621ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:07.171402 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e6d5809f-68b8-4938-9012-7c1c43c621ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:07.178090 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6d5809f-68b8-4938-9012-7c1c43c621ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:07.178608 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6d5809f-68b8-4938-9012-7c1c43c621ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:07.179043 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6d5809f-68b8-4938-9012-7c1c43c621ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:07.179742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6d5809f-68b8-4938-9012-7c1c43c621ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:07.180190 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6d5809f-68b8-4938-9012-7c1c43c621ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:07.180612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6d5809f-68b8-4938-9012-7c1c43c621ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:07.197447 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:07.211026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e6d5809f-68b8-4938-9012-7c1c43c621ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045978 Sep 08 14:03:07.214430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 705/1515] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:07 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:07.771259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-75e99628-8327-48c9-a179-da7376e5adcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:07.772041 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-75e99628-8327-48c9-a179-da7376e5adcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:07.778494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75e99628-8327-48c9-a179-da7376e5adcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:07.778981 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75e99628-8327-48c9-a179-da7376e5adcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:07.779421 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75e99628-8327-48c9-a179-da7376e5adcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:07.780391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75e99628-8327-48c9-a179-da7376e5adcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:07.780864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75e99628-8327-48c9-a179-da7376e5adcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:07.781290 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75e99628-8327-48c9-a179-da7376e5adcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:07.807691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-24dc7ab7-f57d-431a-a3c2-95de6a9ef587 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:07.811036 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-24dc7ab7-f57d-431a-a3c2-95de6a9ef587 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:07.817344 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-24dc7ab7-f57d-431a-a3c2-95de6a9ef587 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:07.817843 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-24dc7ab7-f57d-431a-a3c2-95de6a9ef587 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:07.818319 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-24dc7ab7-f57d-431a-a3c2-95de6a9ef587 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:07.819022 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-24dc7ab7-f57d-431a-a3c2-95de6a9ef587 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:07.819542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-24dc7ab7-f57d-431a-a3c2-95de6a9ef587 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:07.819972 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-24dc7ab7-f57d-431a-a3c2-95de6a9ef587 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:07.845870 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-75e99628-8327-48c9-a179-da7376e5adcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:07.853413 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-75e99628-8327-48c9-a179-da7376e5adcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.087210 Sep 08 14:03:07.853594 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-24dc7ab7-f57d-431a-a3c2-95de6a9ef587 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050198 Sep 08 14:03:07.853977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 811/1516] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:07 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:07.858501 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 706/1517] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:07 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:07.879959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6ffff868-ea39-40a4-b35d-8a32d98ba24d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:07.880566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6ffff868-ea39-40a4-b35d-8a32d98ba24d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:07.886506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ffff868-ea39-40a4-b35d-8a32d98ba24d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:07.886992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ffff868-ea39-40a4-b35d-8a32d98ba24d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:07.887431 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ffff868-ea39-40a4-b35d-8a32d98ba24d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:07.888209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ffff868-ea39-40a4-b35d-8a32d98ba24d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:07.888609 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ffff868-ea39-40a4-b35d-8a32d98ba24d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:07.889036 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ffff868-ea39-40a4-b35d-8a32d98ba24d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:07.968275 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-49b90bf9-080e-4dc9-88e8-fc7c3cbf61cc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:07.970452 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-49b90bf9-080e-4dc9-88e8-fc7c3cbf61cc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:07.976406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-49b90bf9-080e-4dc9-88e8-fc7c3cbf61cc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:07.976887 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-49b90bf9-080e-4dc9-88e8-fc7c3cbf61cc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:07.977325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-49b90bf9-080e-4dc9-88e8-fc7c3cbf61cc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:07.978013 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-49b90bf9-080e-4dc9-88e8-fc7c3cbf61cc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:07.978484 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-49b90bf9-080e-4dc9-88e8-fc7c3cbf61cc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:07.978903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-49b90bf9-080e-4dc9-88e8-fc7c3cbf61cc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:08.009512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-49b90bf9-080e-4dc9-88e8-fc7c3cbf61cc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045304 Sep 08 14:03:08.010088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 707/1518] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:07 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:08.028195 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6ffff868-ea39-40a4-b35d-8a32d98ba24d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:08.031839 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6ffff868-ea39-40a4-b35d-8a32d98ba24d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.156210 Sep 08 14:03:08.032427 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 812/1519] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:07 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:08.225219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c09469f5-5ade-4ee2-a95c-03d5c4e13f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:08.227386 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c09469f5-5ade-4ee2-a95c-03d5c4e13f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:08.233701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c09469f5-5ade-4ee2-a95c-03d5c4e13f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:08.234209 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c09469f5-5ade-4ee2-a95c-03d5c4e13f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:08.234650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c09469f5-5ade-4ee2-a95c-03d5c4e13f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:08.235352 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c09469f5-5ade-4ee2-a95c-03d5c4e13f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:08.235829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c09469f5-5ade-4ee2-a95c-03d5c4e13f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:08.236360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c09469f5-5ade-4ee2-a95c-03d5c4e13f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:08.266844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c09469f5-5ade-4ee2-a95c-03d5c4e13f21 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045715 Sep 08 14:03:08.267410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 708/1520] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:08 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:08.867108 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e8676d9c-fd0e-42b8-bed1-f4ec2f5516dc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:08.869271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e8676d9c-fd0e-42b8-bed1-f4ec2f5516dc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:08.876202 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e8676d9c-fd0e-42b8-bed1-f4ec2f5516dc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:08.876375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5b7bfe16-de8e-45c6-a6e7-8a5fe83eaeeb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:08.876804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e8676d9c-fd0e-42b8-bed1-f4ec2f5516dc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:08.876902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5b7bfe16-de8e-45c6-a6e7-8a5fe83eaeeb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:08.877126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e8676d9c-fd0e-42b8-bed1-f4ec2f5516dc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:08.878402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e8676d9c-fd0e-42b8-bed1-f4ec2f5516dc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:08.878876 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e8676d9c-fd0e-42b8-bed1-f4ec2f5516dc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:08.879300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e8676d9c-fd0e-42b8-bed1-f4ec2f5516dc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:08.883550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5b7bfe16-de8e-45c6-a6e7-8a5fe83eaeeb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:08.884032 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5b7bfe16-de8e-45c6-a6e7-8a5fe83eaeeb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:08.884472 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5b7bfe16-de8e-45c6-a6e7-8a5fe83eaeeb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:08.885170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5b7bfe16-de8e-45c6-a6e7-8a5fe83eaeeb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:08.885642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5b7bfe16-de8e-45c6-a6e7-8a5fe83eaeeb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:08.886062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5b7bfe16-de8e-45c6-a6e7-8a5fe83eaeeb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:08.910796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e8676d9c-fd0e-42b8-bed1-f4ec2f5516dc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046949 Sep 08 14:03:08.910796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 813/1521] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:08 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:08.950014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5b7bfe16-de8e-45c6-a6e7-8a5fe83eaeeb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:08.958372 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5b7bfe16-de8e-45c6-a6e7-8a5fe83eaeeb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.086120 Sep 08 14:03:08.958938 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 709/1522] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:08 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:09.022945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-039c2ecc-31fd-4bf4-b342-3a5c6aa7b9e8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:09.025225 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-039c2ecc-31fd-4bf4-b342-3a5c6aa7b9e8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:09.031103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-039c2ecc-31fd-4bf4-b342-3a5c6aa7b9e8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:09.031708 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-039c2ecc-31fd-4bf4-b342-3a5c6aa7b9e8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:09.032141 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-039c2ecc-31fd-4bf4-b342-3a5c6aa7b9e8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:09.032832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-039c2ecc-31fd-4bf4-b342-3a5c6aa7b9e8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:09.033280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-039c2ecc-31fd-4bf4-b342-3a5c6aa7b9e8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:09.033701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-039c2ecc-31fd-4bf4-b342-3a5c6aa7b9e8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:09.055328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4d60fe53-89f5-496c-aa79-7153d64194e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:09.056049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4d60fe53-89f5-496c-aa79-7153d64194e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:09.062413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d60fe53-89f5-496c-aa79-7153d64194e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:09.062904 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d60fe53-89f5-496c-aa79-7153d64194e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:09.063484 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d60fe53-89f5-496c-aa79-7153d64194e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:09.064189 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d60fe53-89f5-496c-aa79-7153d64194e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:09.064640 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d60fe53-89f5-496c-aa79-7153d64194e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:09.065063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d60fe53-89f5-496c-aa79-7153d64194e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:09.068475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-039c2ecc-31fd-4bf4-b342-3a5c6aa7b9e8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050392 Sep 08 14:03:09.070070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 814/1523] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:09 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:09.204551 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4d60fe53-89f5-496c-aa79-7153d64194e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:09.213873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4d60fe53-89f5-496c-aa79-7153d64194e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.162516 Sep 08 14:03:09.214487 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 710/1524] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:09 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:09.278799 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3df244bb-932e-473d-bcd5-016d3ab643f4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:09.280978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3df244bb-932e-473d-bcd5-016d3ab643f4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:09.287838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3df244bb-932e-473d-bcd5-016d3ab643f4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:09.288327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3df244bb-932e-473d-bcd5-016d3ab643f4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:09.288769 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3df244bb-932e-473d-bcd5-016d3ab643f4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:09.289468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3df244bb-932e-473d-bcd5-016d3ab643f4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:09.289926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3df244bb-932e-473d-bcd5-016d3ab643f4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:09.290398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3df244bb-932e-473d-bcd5-016d3ab643f4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:09.320455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3df244bb-932e-473d-bcd5-016d3ab643f4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045624 Sep 08 14:03:09.321016 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 815/1525] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:09 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:09.922681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-83b4a1b5-7534-4eaf-a855-8db4efadc8d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:09.925460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-83b4a1b5-7534-4eaf-a855-8db4efadc8d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:09.932222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-83b4a1b5-7534-4eaf-a855-8db4efadc8d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:09.932706 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-83b4a1b5-7534-4eaf-a855-8db4efadc8d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:09.933143 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-83b4a1b5-7534-4eaf-a855-8db4efadc8d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:09.933847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-83b4a1b5-7534-4eaf-a855-8db4efadc8d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:09.934341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-83b4a1b5-7534-4eaf-a855-8db4efadc8d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:09.934770 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-83b4a1b5-7534-4eaf-a855-8db4efadc8d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:09.966158 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-83b4a1b5-7534-4eaf-a855-8db4efadc8d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047431 Sep 08 14:03:09.966762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 711/1526] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:09 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:09.982624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fc45015e-3599-48e1-bad4-ca4584e23c03 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:09.983232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fc45015e-3599-48e1-bad4-ca4584e23c03 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:09.990449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc45015e-3599-48e1-bad4-ca4584e23c03 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:09.990943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc45015e-3599-48e1-bad4-ca4584e23c03 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:09.991390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc45015e-3599-48e1-bad4-ca4584e23c03 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:09.992120 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc45015e-3599-48e1-bad4-ca4584e23c03 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:09.992559 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc45015e-3599-48e1-bad4-ca4584e23c03 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:09.992991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc45015e-3599-48e1-bad4-ca4584e23c03 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:10.055572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fc45015e-3599-48e1-bad4-ca4584e23c03 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:10.063292 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fc45015e-3599-48e1-bad4-ca4584e23c03 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.084575 Sep 08 14:03:10.063865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 816/1527] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:09 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:10.081314 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9e404aba-1c37-42d6-bb19-40f8b7c6bd63 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:10.083562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9e404aba-1c37-42d6-bb19-40f8b7c6bd63 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:10.089604 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e404aba-1c37-42d6-bb19-40f8b7c6bd63 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:10.090106 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e404aba-1c37-42d6-bb19-40f8b7c6bd63 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:10.090560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e404aba-1c37-42d6-bb19-40f8b7c6bd63 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:10.091235 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e404aba-1c37-42d6-bb19-40f8b7c6bd63 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:10.091730 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e404aba-1c37-42d6-bb19-40f8b7c6bd63 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:10.092145 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9e404aba-1c37-42d6-bb19-40f8b7c6bd63 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:10.122226 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9e404aba-1c37-42d6-bb19-40f8b7c6bd63 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045070 Sep 08 14:03:10.122806 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 712/1528] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:10 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:10.238988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d666ac6f-c199-4fcf-b551-213d7f02ffa8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:10.239513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d666ac6f-c199-4fcf-b551-213d7f02ffa8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:10.245621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d666ac6f-c199-4fcf-b551-213d7f02ffa8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:10.246112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d666ac6f-c199-4fcf-b551-213d7f02ffa8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:10.246592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d666ac6f-c199-4fcf-b551-213d7f02ffa8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:10.247299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d666ac6f-c199-4fcf-b551-213d7f02ffa8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:10.247752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d666ac6f-c199-4fcf-b551-213d7f02ffa8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:10.248180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d666ac6f-c199-4fcf-b551-213d7f02ffa8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:10.332940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a76be9e0-bbfa-477c-80d6-b117a3c3d433 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:10.335156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a76be9e0-bbfa-477c-80d6-b117a3c3d433 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:10.341157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a76be9e0-bbfa-477c-80d6-b117a3c3d433 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:10.341635 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a76be9e0-bbfa-477c-80d6-b117a3c3d433 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:10.342062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a76be9e0-bbfa-477c-80d6-b117a3c3d433 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:10.342770 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a76be9e0-bbfa-477c-80d6-b117a3c3d433 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:10.343214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a76be9e0-bbfa-477c-80d6-b117a3c3d433 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:10.343630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a76be9e0-bbfa-477c-80d6-b117a3c3d433 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:10.374986 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a76be9e0-bbfa-477c-80d6-b117a3c3d433 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045965 Sep 08 14:03:10.375557 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 713/1529] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:10 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:03:10.381997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d666ac6f-c199-4fcf-b551-213d7f02ffa8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:10.389912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d666ac6f-c199-4fcf-b551-213d7f02ffa8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.155676 Sep 08 14:03:10.390514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 817/1530] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:10 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:10.978491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2d7ab8bf-7277-4061-86fc-d9751958a1f0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:10.980567 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2d7ab8bf-7277-4061-86fc-d9751958a1f0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:10.987210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2d7ab8bf-7277-4061-86fc-d9751958a1f0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:10.987690 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2d7ab8bf-7277-4061-86fc-d9751958a1f0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:10.988127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2d7ab8bf-7277-4061-86fc-d9751958a1f0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:10.988815 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2d7ab8bf-7277-4061-86fc-d9751958a1f0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:10.989265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2d7ab8bf-7277-4061-86fc-d9751958a1f0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:10.989745 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2d7ab8bf-7277-4061-86fc-d9751958a1f0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:11.023666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2d7ab8bf-7277-4061-86fc-d9751958a1f0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049220 Sep 08 14:03:11.024246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 714/1531] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:10 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:11.087121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fb10d7d3-72b1-4f59-8d68-a624fe997a2e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:11.087728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fb10d7d3-72b1-4f59-8d68-a624fe997a2e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:11.094278 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb10d7d3-72b1-4f59-8d68-a624fe997a2e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:11.094768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb10d7d3-72b1-4f59-8d68-a624fe997a2e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:11.095206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb10d7d3-72b1-4f59-8d68-a624fe997a2e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:11.095894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb10d7d3-72b1-4f59-8d68-a624fe997a2e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:11.096430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb10d7d3-72b1-4f59-8d68-a624fe997a2e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:11.096765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb10d7d3-72b1-4f59-8d68-a624fe997a2e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:11.134754 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1b0ced2a-1e2d-4c82-8a9e-a77a04cebe65 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:11.136978 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1b0ced2a-1e2d-4c82-8a9e-a77a04cebe65 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:11.143625 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b0ced2a-1e2d-4c82-8a9e-a77a04cebe65 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:11.144116 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b0ced2a-1e2d-4c82-8a9e-a77a04cebe65 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:11.144553 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b0ced2a-1e2d-4c82-8a9e-a77a04cebe65 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:11.145904 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b0ced2a-1e2d-4c82-8a9e-a77a04cebe65 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:11.146384 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b0ced2a-1e2d-4c82-8a9e-a77a04cebe65 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:11.146808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b0ced2a-1e2d-4c82-8a9e-a77a04cebe65 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:11.164742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fb10d7d3-72b1-4f59-8d68-a624fe997a2e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:11.173148 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fb10d7d3-72b1-4f59-8d68-a624fe997a2e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.089691 Sep 08 14:03:11.174029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 818/1532] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:11 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:11.177245 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1b0ced2a-1e2d-4c82-8a9e-a77a04cebe65 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046371 Sep 08 14:03:11.177810 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 715/1533] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:11 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:11.387224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-508a1e6c-4b5f-44f4-995c-a072d948d443 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:11.389418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-508a1e6c-4b5f-44f4-995c-a072d948d443 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:11.396387 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-508a1e6c-4b5f-44f4-995c-a072d948d443 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:11.396877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-508a1e6c-4b5f-44f4-995c-a072d948d443 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:11.397318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-508a1e6c-4b5f-44f4-995c-a072d948d443 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:11.398027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-508a1e6c-4b5f-44f4-995c-a072d948d443 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:11.398520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-508a1e6c-4b5f-44f4-995c-a072d948d443 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:11.398953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-508a1e6c-4b5f-44f4-995c-a072d948d443 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:11.413959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-976353f5-d14f-40d3-abde-e0f0c3859071 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:11.414608 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-976353f5-d14f-40d3-abde-e0f0c3859071 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:11.416466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:11.421603 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-976353f5-d14f-40d3-abde-e0f0c3859071 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:11.422098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-976353f5-d14f-40d3-abde-e0f0c3859071 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:11.422565 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-976353f5-d14f-40d3-abde-e0f0c3859071 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:11.423273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-976353f5-d14f-40d3-abde-e0f0c3859071 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:11.423727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-976353f5-d14f-40d3-abde-e0f0c3859071 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:11.424155 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-976353f5-d14f-40d3-abde-e0f0c3859071 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:11.430901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-508a1e6c-4b5f-44f4-995c-a072d948d443 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047734 Sep 08 14:03:11.431476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 819/1534] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:11 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:11.560760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-976353f5-d14f-40d3-abde-e0f0c3859071 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:11.569326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-976353f5-d14f-40d3-abde-e0f0c3859071 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.159437 Sep 08 14:03:11.571229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 716/1535] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:11 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:12.036277 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-55d7d49a-3af0-49e3-ac24-fedc62e6d50d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:12.038442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-55d7d49a-3af0-49e3-ac24-fedc62e6d50d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:12.044463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-55d7d49a-3af0-49e3-ac24-fedc62e6d50d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:12.044952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-55d7d49a-3af0-49e3-ac24-fedc62e6d50d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:12.045387 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-55d7d49a-3af0-49e3-ac24-fedc62e6d50d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:12.046077 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-55d7d49a-3af0-49e3-ac24-fedc62e6d50d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:12.046552 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-55d7d49a-3af0-49e3-ac24-fedc62e6d50d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:12.046976 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-55d7d49a-3af0-49e3-ac24-fedc62e6d50d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:12.077123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-55d7d49a-3af0-49e3-ac24-fedc62e6d50d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044832 Sep 08 14:03:12.077665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 820/1536] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:12 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:12.189302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b901d2db-1721-4538-bb0f-f91a20ac8095 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:12.192174 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b901d2db-1721-4538-bb0f-f91a20ac8095 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:12.197264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-79c42ee4-af55-48b5-ae19-920cecc5bda3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:12.197877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-79c42ee4-af55-48b5-ae19-920cecc5bda3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:12.199002 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b901d2db-1721-4538-bb0f-f91a20ac8095 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:12.199504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b901d2db-1721-4538-bb0f-f91a20ac8095 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:12.199943 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b901d2db-1721-4538-bb0f-f91a20ac8095 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:12.200652 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b901d2db-1721-4538-bb0f-f91a20ac8095 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:12.201103 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b901d2db-1721-4538-bb0f-f91a20ac8095 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:12.201529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b901d2db-1721-4538-bb0f-f91a20ac8095 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:12.204689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-79c42ee4-af55-48b5-ae19-920cecc5bda3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:12.205181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-79c42ee4-af55-48b5-ae19-920cecc5bda3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:12.205624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-79c42ee4-af55-48b5-ae19-920cecc5bda3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:12.206500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-79c42ee4-af55-48b5-ae19-920cecc5bda3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:12.206974 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-79c42ee4-af55-48b5-ae19-920cecc5bda3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:12.207407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-79c42ee4-af55-48b5-ae19-920cecc5bda3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:12.237448 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b901d2db-1721-4538-bb0f-f91a20ac8095 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051783 Sep 08 14:03:12.238024 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 717/1537] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:12 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:12.273440 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-79c42ee4-af55-48b5-ae19-920cecc5bda3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:12.281333 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-79c42ee4-af55-48b5-ae19-920cecc5bda3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.087883 Sep 08 14:03:12.281900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 821/1538] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:12 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:12.457198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-213ca8ae-8411-4128-ad45-0241376e86cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:12.459330 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-213ca8ae-8411-4128-ad45-0241376e86cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:12.465863 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-213ca8ae-8411-4128-ad45-0241376e86cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:12.466379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-213ca8ae-8411-4128-ad45-0241376e86cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:12.466815 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-213ca8ae-8411-4128-ad45-0241376e86cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:12.467507 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-213ca8ae-8411-4128-ad45-0241376e86cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:12.467947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-213ca8ae-8411-4128-ad45-0241376e86cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:12.468356 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-213ca8ae-8411-4128-ad45-0241376e86cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:12.500563 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-213ca8ae-8411-4128-ad45-0241376e86cb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047449 Sep 08 14:03:12.501130 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 718/1539] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:12 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:12.594902 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-be369657-c908-4cbe-b7d4-4a6f86e89c51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:12.595513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-be369657-c908-4cbe-b7d4-4a6f86e89c51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:12.602164 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be369657-c908-4cbe-b7d4-4a6f86e89c51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:12.602656 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be369657-c908-4cbe-b7d4-4a6f86e89c51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:12.603100 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be369657-c908-4cbe-b7d4-4a6f86e89c51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:12.603788 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be369657-c908-4cbe-b7d4-4a6f86e89c51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:12.604320 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be369657-c908-4cbe-b7d4-4a6f86e89c51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:12.606366 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be369657-c908-4cbe-b7d4-4a6f86e89c51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:12.740137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-be369657-c908-4cbe-b7d4-4a6f86e89c51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:12.750649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-be369657-c908-4cbe-b7d4-4a6f86e89c51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.159238 Sep 08 14:03:12.751516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 822/1540] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:12 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:13.089466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5d1cf991-cefe-480d-8e26-557f30c7b9b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:13.091667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5d1cf991-cefe-480d-8e26-557f30c7b9b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:13.097640 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d1cf991-cefe-480d-8e26-557f30c7b9b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:13.098148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d1cf991-cefe-480d-8e26-557f30c7b9b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:13.098602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d1cf991-cefe-480d-8e26-557f30c7b9b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:13.099293 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d1cf991-cefe-480d-8e26-557f30c7b9b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:13.099746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d1cf991-cefe-480d-8e26-557f30c7b9b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:13.100170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d1cf991-cefe-480d-8e26-557f30c7b9b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:13.129936 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5d1cf991-cefe-480d-8e26-557f30c7b9b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044470 Sep 08 14:03:13.130518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 719/1541] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:13 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:13.250516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cac12174-50bf-4e2a-9482-4459fe973c94 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:13.253159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cac12174-50bf-4e2a-9482-4459fe973c94 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:13.259618 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cac12174-50bf-4e2a-9482-4459fe973c94 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:13.260100 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cac12174-50bf-4e2a-9482-4459fe973c94 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:13.260538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cac12174-50bf-4e2a-9482-4459fe973c94 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:13.261232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cac12174-50bf-4e2a-9482-4459fe973c94 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:13.261693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cac12174-50bf-4e2a-9482-4459fe973c94 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:13.262118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cac12174-50bf-4e2a-9482-4459fe973c94 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:13.291357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cac12174-50bf-4e2a-9482-4459fe973c94 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045111 Sep 08 14:03:13.291912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 823/1542] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:13 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:13.304353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b3d8f498-1605-4f29-a380-bd0e363c2b16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:13.304958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b3d8f498-1605-4f29-a380-bd0e363c2b16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:13.311330 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3d8f498-1605-4f29-a380-bd0e363c2b16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:13.311811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3d8f498-1605-4f29-a380-bd0e363c2b16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:13.312241 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3d8f498-1605-4f29-a380-bd0e363c2b16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:13.312928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3d8f498-1605-4f29-a380-bd0e363c2b16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:13.313384 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3d8f498-1605-4f29-a380-bd0e363c2b16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:13.313802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3d8f498-1605-4f29-a380-bd0e363c2b16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:13.376034 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b3d8f498-1605-4f29-a380-bd0e363c2b16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:13.384312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b3d8f498-1605-4f29-a380-bd0e363c2b16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.083804 Sep 08 14:03:13.384881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 720/1543] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:13 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:13.513493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4528f256-e7f6-4093-b3ee-c1d64b970b95 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:13.515644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4528f256-e7f6-4093-b3ee-c1d64b970b95 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:13.521547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4528f256-e7f6-4093-b3ee-c1d64b970b95 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:13.522035 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4528f256-e7f6-4093-b3ee-c1d64b970b95 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:13.522513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4528f256-e7f6-4093-b3ee-c1d64b970b95 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:13.523199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4528f256-e7f6-4093-b3ee-c1d64b970b95 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:13.523650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4528f256-e7f6-4093-b3ee-c1d64b970b95 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:13.524074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4528f256-e7f6-4093-b3ee-c1d64b970b95 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:13.553199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4528f256-e7f6-4093-b3ee-c1d64b970b95 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.043916 Sep 08 14:03:13.553749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 824/1544] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:13 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:03:13.774480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0adfba1c-dea1-4f21-a59d-711e742a4cce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:13.775086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0adfba1c-dea1-4f21-a59d-711e742a4cce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:13.781153 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0adfba1c-dea1-4f21-a59d-711e742a4cce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:13.781636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0adfba1c-dea1-4f21-a59d-711e742a4cce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:13.782069 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0adfba1c-dea1-4f21-a59d-711e742a4cce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:13.782778 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0adfba1c-dea1-4f21-a59d-711e742a4cce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:13.783233 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0adfba1c-dea1-4f21-a59d-711e742a4cce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:13.783887 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0adfba1c-dea1-4f21-a59d-711e742a4cce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:13.921880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0adfba1c-dea1-4f21-a59d-711e742a4cce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:13.930381 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0adfba1c-dea1-4f21-a59d-711e742a4cce tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.159806 Sep 08 14:03:13.930957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 721/1545] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:13 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:14.142714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c75e0aec-2091-4470-ad22-d5ddd2858f00 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:14.144966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c75e0aec-2091-4470-ad22-d5ddd2858f00 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:14.150987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c75e0aec-2091-4470-ad22-d5ddd2858f00 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:14.151479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c75e0aec-2091-4470-ad22-d5ddd2858f00 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:14.151925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c75e0aec-2091-4470-ad22-d5ddd2858f00 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:14.152619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c75e0aec-2091-4470-ad22-d5ddd2858f00 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:14.153079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c75e0aec-2091-4470-ad22-d5ddd2858f00 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:14.153507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c75e0aec-2091-4470-ad22-d5ddd2858f00 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:14.184233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c75e0aec-2091-4470-ad22-d5ddd2858f00 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045637 Sep 08 14:03:14.184748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 825/1546] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:14 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:14.303741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f01aa8fa-0636-4c4f-a7d1-92c101d406fb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:14.305893 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f01aa8fa-0636-4c4f-a7d1-92c101d406fb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:14.312413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f01aa8fa-0636-4c4f-a7d1-92c101d406fb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:14.312897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f01aa8fa-0636-4c4f-a7d1-92c101d406fb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:14.313339 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f01aa8fa-0636-4c4f-a7d1-92c101d406fb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:14.314030 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f01aa8fa-0636-4c4f-a7d1-92c101d406fb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:14.314603 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f01aa8fa-0636-4c4f-a7d1-92c101d406fb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:14.315027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f01aa8fa-0636-4c4f-a7d1-92c101d406fb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:14.318961 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:14.348820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f01aa8fa-0636-4c4f-a7d1-92c101d406fb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049001 Sep 08 14:03:14.349391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 722/1547] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:14 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:14.411841 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8a1c38c8-fed0-4922-9963-67fde231459e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:14.412811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8a1c38c8-fed0-4922-9963-67fde231459e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:14.422157 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a1c38c8-fed0-4922-9963-67fde231459e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:14.422902 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a1c38c8-fed0-4922-9963-67fde231459e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:14.423568 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a1c38c8-fed0-4922-9963-67fde231459e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:14.424630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a1c38c8-fed0-4922-9963-67fde231459e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:14.425332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a1c38c8-fed0-4922-9963-67fde231459e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:14.426031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a1c38c8-fed0-4922-9963-67fde231459e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:14.432085 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:14.495852 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8a1c38c8-fed0-4922-9963-67fde231459e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:14.503856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8a1c38c8-fed0-4922-9963-67fde231459e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.098793 Sep 08 14:03:14.504418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 826/1548] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:14 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:14.565690 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-643eb66d-b7d6-4ed9-90a5-a5c6b6ac9fd0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:14.567909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-643eb66d-b7d6-4ed9-90a5-a5c6b6ac9fd0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:14.574282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-643eb66d-b7d6-4ed9-90a5-a5c6b6ac9fd0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:14.574763 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-643eb66d-b7d6-4ed9-90a5-a5c6b6ac9fd0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:14.575194 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-643eb66d-b7d6-4ed9-90a5-a5c6b6ac9fd0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:14.575879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-643eb66d-b7d6-4ed9-90a5-a5c6b6ac9fd0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:14.576338 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-643eb66d-b7d6-4ed9-90a5-a5c6b6ac9fd0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:14.576746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-643eb66d-b7d6-4ed9-90a5-a5c6b6ac9fd0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:14.607200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-643eb66d-b7d6-4ed9-90a5-a5c6b6ac9fd0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046165 Sep 08 14:03:14.607732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 723/1549] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:14 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:14.952938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e78a8541-14bc-4223-8514-74399096102c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:14.953539 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e78a8541-14bc-4223-8514-74399096102c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:14.959525 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e78a8541-14bc-4223-8514-74399096102c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:14.960008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e78a8541-14bc-4223-8514-74399096102c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:14.960451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e78a8541-14bc-4223-8514-74399096102c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:14.961139 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e78a8541-14bc-4223-8514-74399096102c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:14.961591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e78a8541-14bc-4223-8514-74399096102c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:14.962012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e78a8541-14bc-4223-8514-74399096102c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:15.095365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e78a8541-14bc-4223-8514-74399096102c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:15.103352 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e78a8541-14bc-4223-8514-74399096102c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.154306 Sep 08 14:03:15.103921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 827/1550] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:14 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:15.197049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ba857b5b-3eb8-4165-9e17-ae662aa2a628 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:15.199262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ba857b5b-3eb8-4165-9e17-ae662aa2a628 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:15.205306 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ba857b5b-3eb8-4165-9e17-ae662aa2a628 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:15.205741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ba857b5b-3eb8-4165-9e17-ae662aa2a628 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:15.206175 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ba857b5b-3eb8-4165-9e17-ae662aa2a628 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:15.206860 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ba857b5b-3eb8-4165-9e17-ae662aa2a628 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:15.207310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ba857b5b-3eb8-4165-9e17-ae662aa2a628 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:15.207731 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ba857b5b-3eb8-4165-9e17-ae662aa2a628 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:15.237469 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ba857b5b-3eb8-4165-9e17-ae662aa2a628 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044293 Sep 08 14:03:15.238086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 724/1551] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:15 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:15.362406 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4759458a-05d8-455f-9c7c-4ed6aacfc6f7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:15.364564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4759458a-05d8-455f-9c7c-4ed6aacfc6f7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:15.370718 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4759458a-05d8-455f-9c7c-4ed6aacfc6f7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:15.371200 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4759458a-05d8-455f-9c7c-4ed6aacfc6f7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:15.371641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4759458a-05d8-455f-9c7c-4ed6aacfc6f7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:15.372341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4759458a-05d8-455f-9c7c-4ed6aacfc6f7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:15.372793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4759458a-05d8-455f-9c7c-4ed6aacfc6f7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:15.373471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4759458a-05d8-455f-9c7c-4ed6aacfc6f7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:15.402875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4759458a-05d8-455f-9c7c-4ed6aacfc6f7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045114 Sep 08 14:03:15.403442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 828/1552] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:15 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:15.528912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0d4fb703-988b-4293-a302-16d77e067100 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:15.529438 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0d4fb703-988b-4293-a302-16d77e067100 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:15.535966 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0d4fb703-988b-4293-a302-16d77e067100 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:15.536446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0d4fb703-988b-4293-a302-16d77e067100 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:15.536880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0d4fb703-988b-4293-a302-16d77e067100 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:15.537566 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0d4fb703-988b-4293-a302-16d77e067100 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:15.538019 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0d4fb703-988b-4293-a302-16d77e067100 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:15.538476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0d4fb703-988b-4293-a302-16d77e067100 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:15.602090 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0d4fb703-988b-4293-a302-16d77e067100 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:15.610363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0d4fb703-988b-4293-a302-16d77e067100 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.085316 Sep 08 14:03:15.610942 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 725/1553] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:15 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:15.619897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8fc88e1d-da92-4047-8a00-d5eea68cd355 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:15.622034 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8fc88e1d-da92-4047-8a00-d5eea68cd355 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:15.629707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fc88e1d-da92-4047-8a00-d5eea68cd355 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:15.630223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fc88e1d-da92-4047-8a00-d5eea68cd355 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:15.630665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fc88e1d-da92-4047-8a00-d5eea68cd355 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:15.631516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fc88e1d-da92-4047-8a00-d5eea68cd355 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:15.631973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fc88e1d-da92-4047-8a00-d5eea68cd355 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:15.632399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fc88e1d-da92-4047-8a00-d5eea68cd355 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:15.664718 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8fc88e1d-da92-4047-8a00-d5eea68cd355 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048738 Sep 08 14:03:15.665308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 829/1554] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:15 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:16.127381 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-abaf2206-c5eb-4ae7-b112-0239a8b60569 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:16.128025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-abaf2206-c5eb-4ae7-b112-0239a8b60569 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:16.134049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abaf2206-c5eb-4ae7-b112-0239a8b60569 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:16.134565 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abaf2206-c5eb-4ae7-b112-0239a8b60569 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:16.135001 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abaf2206-c5eb-4ae7-b112-0239a8b60569 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:16.135694 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abaf2206-c5eb-4ae7-b112-0239a8b60569 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:16.136286 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abaf2206-c5eb-4ae7-b112-0239a8b60569 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:16.136727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abaf2206-c5eb-4ae7-b112-0239a8b60569 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:16.250989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0133a1c3-0296-4079-8a41-30456d24196b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:16.253381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0133a1c3-0296-4079-8a41-30456d24196b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:16.259489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0133a1c3-0296-4079-8a41-30456d24196b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:16.259978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0133a1c3-0296-4079-8a41-30456d24196b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:16.260421 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0133a1c3-0296-4079-8a41-30456d24196b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:16.261116 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0133a1c3-0296-4079-8a41-30456d24196b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:16.261570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0133a1c3-0296-4079-8a41-30456d24196b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:16.261997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0133a1c3-0296-4079-8a41-30456d24196b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:16.270723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-abaf2206-c5eb-4ae7-b112-0239a8b60569 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:16.278462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-abaf2206-c5eb-4ae7-b112-0239a8b60569 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.155288 Sep 08 14:03:16.279039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 726/1555] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:16 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:16.292314 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0133a1c3-0296-4079-8a41-30456d24196b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045803 Sep 08 14:03:16.292864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 830/1556] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:16 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:16.415480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2c9a3299-4be2-4f45-aa83-818b2e57bd7d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:16.417637 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2c9a3299-4be2-4f45-aa83-818b2e57bd7d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:16.425197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2c9a3299-4be2-4f45-aa83-818b2e57bd7d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:16.425716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2c9a3299-4be2-4f45-aa83-818b2e57bd7d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:16.426161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2c9a3299-4be2-4f45-aa83-818b2e57bd7d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:16.426885 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2c9a3299-4be2-4f45-aa83-818b2e57bd7d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:16.427559 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2c9a3299-4be2-4f45-aa83-818b2e57bd7d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:16.427776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2c9a3299-4be2-4f45-aa83-818b2e57bd7d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:16.462154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2c9a3299-4be2-4f45-aa83-818b2e57bd7d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050725 Sep 08 14:03:16.464644 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 727/1557] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:16 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:16.634459 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bb110d01-8a23-4d9c-9fa3-164998c4a35e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:16.635075 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bb110d01-8a23-4d9c-9fa3-164998c4a35e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:16.641815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb110d01-8a23-4d9c-9fa3-164998c4a35e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:16.642866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb110d01-8a23-4d9c-9fa3-164998c4a35e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:16.643356 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb110d01-8a23-4d9c-9fa3-164998c4a35e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:16.644055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb110d01-8a23-4d9c-9fa3-164998c4a35e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:16.644511 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb110d01-8a23-4d9c-9fa3-164998c4a35e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:16.644943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb110d01-8a23-4d9c-9fa3-164998c4a35e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:16.677191 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b10485a6-b47c-4bab-bbe1-b21dfc5dd1a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:16.679501 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b10485a6-b47c-4bab-bbe1-b21dfc5dd1a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:16.685991 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b10485a6-b47c-4bab-bbe1-b21dfc5dd1a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:16.686483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b10485a6-b47c-4bab-bbe1-b21dfc5dd1a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:16.686916 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b10485a6-b47c-4bab-bbe1-b21dfc5dd1a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:16.687609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b10485a6-b47c-4bab-bbe1-b21dfc5dd1a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:16.688119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b10485a6-b47c-4bab-bbe1-b21dfc5dd1a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:16.688544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b10485a6-b47c-4bab-bbe1-b21dfc5dd1a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:16.709194 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bb110d01-8a23-4d9c-9fa3-164998c4a35e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:16.716931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bb110d01-8a23-4d9c-9fa3-164998c4a35e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.086504 Sep 08 14:03:16.717504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 831/1558] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:16 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:16.718545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b10485a6-b47c-4bab-bbe1-b21dfc5dd1a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045378 Sep 08 14:03:16.719098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 728/1559] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:16 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:03:17.305657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9980a60f-2611-44e6-af83-5c9030de4039 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:17.306707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9980a60f-2611-44e6-af83-5c9030de4039 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:17.308737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-54a2094c-aacf-4fef-a6aa-a96abeab6b99 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:17.312088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-54a2094c-aacf-4fef-a6aa-a96abeab6b99 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:17.317014 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9980a60f-2611-44e6-af83-5c9030de4039 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:17.317543 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9980a60f-2611-44e6-af83-5c9030de4039 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:17.318288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9980a60f-2611-44e6-af83-5c9030de4039 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:17.319403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9980a60f-2611-44e6-af83-5c9030de4039 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:17.320215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9980a60f-2611-44e6-af83-5c9030de4039 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:17.320896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9980a60f-2611-44e6-af83-5c9030de4039 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:17.322087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-54a2094c-aacf-4fef-a6aa-a96abeab6b99 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:17.322905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-54a2094c-aacf-4fef-a6aa-a96abeab6b99 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:17.323659 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-54a2094c-aacf-4fef-a6aa-a96abeab6b99 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:17.324781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-54a2094c-aacf-4fef-a6aa-a96abeab6b99 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:17.325533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-54a2094c-aacf-4fef-a6aa-a96abeab6b99 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:17.326267 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-54a2094c-aacf-4fef-a6aa-a96abeab6b99 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:17.363068 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-54a2094c-aacf-4fef-a6aa-a96abeab6b99 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060911 Sep 08 14:03:17.363758 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 729/1560] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:17 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:17.466334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9980a60f-2611-44e6-af83-5c9030de4039 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:17.474682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9980a60f-2611-44e6-af83-5c9030de4039 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.176403 Sep 08 14:03:17.475295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 832/1561] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:17 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:17.476377 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-89006cc5-b76d-46d2-99bf-a016f2a1ee19 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:17.478502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-89006cc5-b76d-46d2-99bf-a016f2a1ee19 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:17.484586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89006cc5-b76d-46d2-99bf-a016f2a1ee19 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:17.485066 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89006cc5-b76d-46d2-99bf-a016f2a1ee19 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:17.485501 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89006cc5-b76d-46d2-99bf-a016f2a1ee19 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:17.486731 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89006cc5-b76d-46d2-99bf-a016f2a1ee19 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:17.487215 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89006cc5-b76d-46d2-99bf-a016f2a1ee19 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:17.487634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-89006cc5-b76d-46d2-99bf-a016f2a1ee19 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:17.518989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-89006cc5-b76d-46d2-99bf-a016f2a1ee19 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046629 Sep 08 14:03:17.520003 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 730/1562] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:17 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:17.731547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bcb24633-af8e-41b1-994b-98feae937101 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:17.733818 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bcb24633-af8e-41b1-994b-98feae937101 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:17.741118 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0437ca08-84d1-470e-8773-3572a674629c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:17.741643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcb24633-af8e-41b1-994b-98feae937101 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:17.741759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0437ca08-84d1-470e-8773-3572a674629c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:17.741854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcb24633-af8e-41b1-994b-98feae937101 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:17.742193 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcb24633-af8e-41b1-994b-98feae937101 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:17.742888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcb24633-af8e-41b1-994b-98feae937101 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:17.743335 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcb24633-af8e-41b1-994b-98feae937101 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:17.743756 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcb24633-af8e-41b1-994b-98feae937101 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:17.748164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0437ca08-84d1-470e-8773-3572a674629c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:17.748646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0437ca08-84d1-470e-8773-3572a674629c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:17.749650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0437ca08-84d1-470e-8773-3572a674629c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:17.749785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0437ca08-84d1-470e-8773-3572a674629c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:17.750220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0437ca08-84d1-470e-8773-3572a674629c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:17.750748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0437ca08-84d1-470e-8773-3572a674629c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:17.775919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bcb24633-af8e-41b1-994b-98feae937101 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048945 Sep 08 14:03:17.776838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 833/1563] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:17 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:17.824962 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0437ca08-84d1-470e-8773-3572a674629c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:17.835786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0437ca08-84d1-470e-8773-3572a674629c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.098485 Sep 08 14:03:17.836832 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 731/1564] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:17 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:18.375968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6ccf20e9-7d35-4142-9980-e45acb1695b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:18.378658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6ccf20e9-7d35-4142-9980-e45acb1695b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:18.384714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ccf20e9-7d35-4142-9980-e45acb1695b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:18.385226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ccf20e9-7d35-4142-9980-e45acb1695b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:18.385669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ccf20e9-7d35-4142-9980-e45acb1695b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:18.386389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ccf20e9-7d35-4142-9980-e45acb1695b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:18.386848 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ccf20e9-7d35-4142-9980-e45acb1695b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:18.387277 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ccf20e9-7d35-4142-9980-e45acb1695b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:18.416451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6ccf20e9-7d35-4142-9980-e45acb1695b2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044693 Sep 08 14:03:18.417001 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 834/1565] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:18 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:18.509699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d736d354-6454-4e11-b21b-c6e1222879d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:18.510361 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d736d354-6454-4e11-b21b-c6e1222879d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:18.516415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d736d354-6454-4e11-b21b-c6e1222879d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:18.516918 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d736d354-6454-4e11-b21b-c6e1222879d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:18.518036 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d736d354-6454-4e11-b21b-c6e1222879d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:18.518821 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d736d354-6454-4e11-b21b-c6e1222879d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:18.519276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d736d354-6454-4e11-b21b-c6e1222879d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:18.519696 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d736d354-6454-4e11-b21b-c6e1222879d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:18.532053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-634fd2aa-012a-4582-9a8c-9531a797da5c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:18.534190 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-634fd2aa-012a-4582-9a8c-9531a797da5c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:18.540107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-634fd2aa-012a-4582-9a8c-9531a797da5c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:18.540600 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-634fd2aa-012a-4582-9a8c-9531a797da5c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:18.541041 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-634fd2aa-012a-4582-9a8c-9531a797da5c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:18.541745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-634fd2aa-012a-4582-9a8c-9531a797da5c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:18.542233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-634fd2aa-012a-4582-9a8c-9531a797da5c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:18.542667 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-634fd2aa-012a-4582-9a8c-9531a797da5c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:18.578127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-634fd2aa-012a-4582-9a8c-9531a797da5c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049969 Sep 08 14:03:18.578977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 835/1566] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:18 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:18.663644 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d736d354-6454-4e11-b21b-c6e1222879d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:18.671376 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d736d354-6454-4e11-b21b-c6e1222879d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.165528 Sep 08 14:03:18.671936 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 732/1567] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:18 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:18.788819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e7909c43-72d8-4f40-afbf-32f767e32dc5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:18.790962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e7909c43-72d8-4f40-afbf-32f767e32dc5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:18.797310 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e7909c43-72d8-4f40-afbf-32f767e32dc5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:18.797794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e7909c43-72d8-4f40-afbf-32f767e32dc5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:18.798265 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e7909c43-72d8-4f40-afbf-32f767e32dc5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:18.799535 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e7909c43-72d8-4f40-afbf-32f767e32dc5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:18.799985 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e7909c43-72d8-4f40-afbf-32f767e32dc5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:18.800410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e7909c43-72d8-4f40-afbf-32f767e32dc5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:18.832251 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e7909c43-72d8-4f40-afbf-32f767e32dc5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047589 Sep 08 14:03:18.832806 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 836/1568] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:18 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:18.862719 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c5b7bd7a-05bf-45a7-95e5-fe464e95a3f9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:18.863355 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c5b7bd7a-05bf-45a7-95e5-fe464e95a3f9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:18.869810 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5b7bd7a-05bf-45a7-95e5-fe464e95a3f9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:18.870349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5b7bd7a-05bf-45a7-95e5-fe464e95a3f9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:18.870781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5b7bd7a-05bf-45a7-95e5-fe464e95a3f9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:18.871469 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5b7bd7a-05bf-45a7-95e5-fe464e95a3f9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:18.871915 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5b7bd7a-05bf-45a7-95e5-fe464e95a3f9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:18.872327 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5b7bd7a-05bf-45a7-95e5-fe464e95a3f9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:18.933809 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c5b7bd7a-05bf-45a7-95e5-fe464e95a3f9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:18.942648 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c5b7bd7a-05bf-45a7-95e5-fe464e95a3f9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.084386 Sep 08 14:03:18.943207 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 733/1569] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:18 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:19.429524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-35ea0683-57c7-4585-9616-5acd9bed7cfc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:19.431680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-35ea0683-57c7-4585-9616-5acd9bed7cfc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:19.437626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-35ea0683-57c7-4585-9616-5acd9bed7cfc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:19.438121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-35ea0683-57c7-4585-9616-5acd9bed7cfc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:19.438603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-35ea0683-57c7-4585-9616-5acd9bed7cfc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:19.439307 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-35ea0683-57c7-4585-9616-5acd9bed7cfc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:19.439761 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-35ea0683-57c7-4585-9616-5acd9bed7cfc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:19.440192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-35ea0683-57c7-4585-9616-5acd9bed7cfc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:19.469264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-35ea0683-57c7-4585-9616-5acd9bed7cfc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043776 Sep 08 14:03:19.469833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 837/1570] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:19 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:19.591488 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ba3477af-2741-443e-b0ee-eb36d59d17b4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:19.593727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ba3477af-2741-443e-b0ee-eb36d59d17b4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:19.600572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ba3477af-2741-443e-b0ee-eb36d59d17b4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:19.601058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ba3477af-2741-443e-b0ee-eb36d59d17b4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:19.601494 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ba3477af-2741-443e-b0ee-eb36d59d17b4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:19.602235 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ba3477af-2741-443e-b0ee-eb36d59d17b4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:19.602695 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ba3477af-2741-443e-b0ee-eb36d59d17b4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:19.603123 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ba3477af-2741-443e-b0ee-eb36d59d17b4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:19.632175 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ba3477af-2741-443e-b0ee-eb36d59d17b4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044704 Sep 08 14:03:19.632737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 734/1571] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:19 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:19.696235 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3e841b83-d7c0-4498-b26b-c17334ce712e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:19.696771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3e841b83-d7c0-4498-b26b-c17334ce712e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:19.702632 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3e841b83-d7c0-4498-b26b-c17334ce712e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:19.703110 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3e841b83-d7c0-4498-b26b-c17334ce712e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:19.703543 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3e841b83-d7c0-4498-b26b-c17334ce712e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:19.705046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3e841b83-d7c0-4498-b26b-c17334ce712e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:19.705172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3e841b83-d7c0-4498-b26b-c17334ce712e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:19.705598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3e841b83-d7c0-4498-b26b-c17334ce712e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:19.836450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3e841b83-d7c0-4498-b26b-c17334ce712e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:19.844399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3e841b83-d7c0-4498-b26b-c17334ce712e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.152060 Sep 08 14:03:19.844648 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a6e79f69-0730-43b0-bb1b-153139c56cae tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:19.844966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 838/1572] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:19 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:19.846910 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a6e79f69-0730-43b0-bb1b-153139c56cae tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:19.852780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6e79f69-0730-43b0-bb1b-153139c56cae tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:19.853260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6e79f69-0730-43b0-bb1b-153139c56cae tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:19.853686 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6e79f69-0730-43b0-bb1b-153139c56cae tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:19.854394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6e79f69-0730-43b0-bb1b-153139c56cae tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:19.854838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6e79f69-0730-43b0-bb1b-153139c56cae tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:19.855254 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6e79f69-0730-43b0-bb1b-153139c56cae tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:19.885277 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a6e79f69-0730-43b0-bb1b-153139c56cae tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044516 Sep 08 14:03:19.885898 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 735/1573] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:19 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:03:19.965461 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d966cdff-9079-4d92-b3b4-fc2eb8925e0a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:19.966071 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d966cdff-9079-4d92-b3b4-fc2eb8925e0a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:19.972210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d966cdff-9079-4d92-b3b4-fc2eb8925e0a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:19.977764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d966cdff-9079-4d92-b3b4-fc2eb8925e0a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:19.977764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d966cdff-9079-4d92-b3b4-fc2eb8925e0a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:19.977764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d966cdff-9079-4d92-b3b4-fc2eb8925e0a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:19.977764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d966cdff-9079-4d92-b3b4-fc2eb8925e0a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:19.978117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d966cdff-9079-4d92-b3b4-fc2eb8925e0a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:20.036169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d966cdff-9079-4d92-b3b4-fc2eb8925e0a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:20.043900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d966cdff-9079-4d92-b3b4-fc2eb8925e0a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.082511 Sep 08 14:03:20.044470 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 839/1574] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:19 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:20.482563 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f39c865e-9966-4c39-94ab-2fbe0c3ad829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:20.484753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f39c865e-9966-4c39-94ab-2fbe0c3ad829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:20.491328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f39c865e-9966-4c39-94ab-2fbe0c3ad829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:20.491817 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f39c865e-9966-4c39-94ab-2fbe0c3ad829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:20.492253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f39c865e-9966-4c39-94ab-2fbe0c3ad829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:20.492944 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f39c865e-9966-4c39-94ab-2fbe0c3ad829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:20.493402 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f39c865e-9966-4c39-94ab-2fbe0c3ad829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:20.493828 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f39c865e-9966-4c39-94ab-2fbe0c3ad829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:20.523356 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f39c865e-9966-4c39-94ab-2fbe0c3ad829 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044871 Sep 08 14:03:20.523910 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 736/1575] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:20 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:20.644812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-be16b20b-78eb-423b-a626-6e5dfbe662a2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:20.647016 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-be16b20b-78eb-423b-a626-6e5dfbe662a2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:20.648875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:20.653500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be16b20b-78eb-423b-a626-6e5dfbe662a2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:20.653986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be16b20b-78eb-423b-a626-6e5dfbe662a2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:20.654468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be16b20b-78eb-423b-a626-6e5dfbe662a2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:20.655167 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be16b20b-78eb-423b-a626-6e5dfbe662a2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:20.655626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be16b20b-78eb-423b-a626-6e5dfbe662a2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:20.656098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be16b20b-78eb-423b-a626-6e5dfbe662a2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:20.686276 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-be16b20b-78eb-423b-a626-6e5dfbe662a2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045447 Sep 08 14:03:20.686828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 840/1576] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:20 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:20.868140 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e4420efc-02e7-42f3-9aa5-bc7aba93874f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:20.868768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e4420efc-02e7-42f3-9aa5-bc7aba93874f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:20.875421 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4420efc-02e7-42f3-9aa5-bc7aba93874f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:20.875892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4420efc-02e7-42f3-9aa5-bc7aba93874f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:20.876314 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4420efc-02e7-42f3-9aa5-bc7aba93874f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:20.877182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4420efc-02e7-42f3-9aa5-bc7aba93874f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:20.877438 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4420efc-02e7-42f3-9aa5-bc7aba93874f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:20.877850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4420efc-02e7-42f3-9aa5-bc7aba93874f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:20.897580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6b2678ca-47bf-4b74-92a5-d77004ce93e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:20.899795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6b2678ca-47bf-4b74-92a5-d77004ce93e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:20.906322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b2678ca-47bf-4b74-92a5-d77004ce93e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:20.907144 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b2678ca-47bf-4b74-92a5-d77004ce93e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:20.907295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b2678ca-47bf-4b74-92a5-d77004ce93e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:20.908459 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b2678ca-47bf-4b74-92a5-d77004ce93e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:20.908930 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b2678ca-47bf-4b74-92a5-d77004ce93e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:20.909376 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b2678ca-47bf-4b74-92a5-d77004ce93e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:20.945959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6b2678ca-47bf-4b74-92a5-d77004ce93e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052786 Sep 08 14:03:20.946555 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 841/1577] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:20 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:03:21.026060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e4420efc-02e7-42f3-9aa5-bc7aba93874f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:21.034541 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e4420efc-02e7-42f3-9aa5-bc7aba93874f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.170226 Sep 08 14:03:21.035103 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 737/1578] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:20 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:21.067194 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5cd4139e-9ea4-46ce-9922-73c49e5fa45a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:21.067825 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5cd4139e-9ea4-46ce-9922-73c49e5fa45a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:21.073713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5cd4139e-9ea4-46ce-9922-73c49e5fa45a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:21.074229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5cd4139e-9ea4-46ce-9922-73c49e5fa45a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:21.074697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5cd4139e-9ea4-46ce-9922-73c49e5fa45a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:21.075386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5cd4139e-9ea4-46ce-9922-73c49e5fa45a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:21.075845 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5cd4139e-9ea4-46ce-9922-73c49e5fa45a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:21.076271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5cd4139e-9ea4-46ce-9922-73c49e5fa45a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:21.143261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5cd4139e-9ea4-46ce-9922-73c49e5fa45a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:21.145419 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:21.151805 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5cd4139e-9ea4-46ce-9922-73c49e5fa45a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.088481 Sep 08 14:03:21.152417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 842/1579] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:21 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:21.535881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-96c828cd-883f-49c3-8646-99aeba83bcf1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:21.538008 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-96c828cd-883f-49c3-8646-99aeba83bcf1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:21.544014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-96c828cd-883f-49c3-8646-99aeba83bcf1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:21.544500 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-96c828cd-883f-49c3-8646-99aeba83bcf1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:21.544933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-96c828cd-883f-49c3-8646-99aeba83bcf1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:21.545626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-96c828cd-883f-49c3-8646-99aeba83bcf1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:21.546089 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-96c828cd-883f-49c3-8646-99aeba83bcf1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:21.546523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-96c828cd-883f-49c3-8646-99aeba83bcf1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:21.573798 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:21.578462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-96c828cd-883f-49c3-8646-99aeba83bcf1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046649 Sep 08 14:03:21.579020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 738/1580] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:21 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:21.698302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f9433dab-c1cb-47d8-8cba-f0b278f1b505 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:21.700570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f9433dab-c1cb-47d8-8cba-f0b278f1b505 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:21.707981 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9433dab-c1cb-47d8-8cba-f0b278f1b505 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:21.708467 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9433dab-c1cb-47d8-8cba-f0b278f1b505 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:21.708902 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9433dab-c1cb-47d8-8cba-f0b278f1b505 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:21.709695 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9433dab-c1cb-47d8-8cba-f0b278f1b505 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:21.710178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9433dab-c1cb-47d8-8cba-f0b278f1b505 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:21.710602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9433dab-c1cb-47d8-8cba-f0b278f1b505 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:21.739715 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f9433dab-c1cb-47d8-8cba-f0b278f1b505 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045399 Sep 08 14:03:21.740257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 843/1581] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:21 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:21.959247 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-eaa29ca1-62f5-4b56-9dc0-6bfb23de18f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:21.961445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-eaa29ca1-62f5-4b56-9dc0-6bfb23de18f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:21.968291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eaa29ca1-62f5-4b56-9dc0-6bfb23de18f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:21.968771 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eaa29ca1-62f5-4b56-9dc0-6bfb23de18f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:21.969204 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eaa29ca1-62f5-4b56-9dc0-6bfb23de18f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:21.970262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eaa29ca1-62f5-4b56-9dc0-6bfb23de18f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:21.970493 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eaa29ca1-62f5-4b56-9dc0-6bfb23de18f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:21.970910 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eaa29ca1-62f5-4b56-9dc0-6bfb23de18f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:22.002115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-eaa29ca1-62f5-4b56-9dc0-6bfb23de18f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047601 Sep 08 14:03:22.002694 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 739/1582] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:21 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:22.056877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-caad82fd-51e6-4a48-afb9-5a00c12a9289 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:22.057477 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-caad82fd-51e6-4a48-afb9-5a00c12a9289 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:22.064670 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-caad82fd-51e6-4a48-afb9-5a00c12a9289 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:22.065156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-caad82fd-51e6-4a48-afb9-5a00c12a9289 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:22.065575 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-caad82fd-51e6-4a48-afb9-5a00c12a9289 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:22.066297 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-caad82fd-51e6-4a48-afb9-5a00c12a9289 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:22.066745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-caad82fd-51e6-4a48-afb9-5a00c12a9289 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:22.067161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-caad82fd-51e6-4a48-afb9-5a00c12a9289 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:22.179768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0ba72f6d-87af-4c1d-8bdd-44983087686c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:22.180345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0ba72f6d-87af-4c1d-8bdd-44983087686c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:22.186182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ba72f6d-87af-4c1d-8bdd-44983087686c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:22.186669 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ba72f6d-87af-4c1d-8bdd-44983087686c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:22.187100 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ba72f6d-87af-4c1d-8bdd-44983087686c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:22.187799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ba72f6d-87af-4c1d-8bdd-44983087686c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:22.188393 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ba72f6d-87af-4c1d-8bdd-44983087686c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:22.188801 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ba72f6d-87af-4c1d-8bdd-44983087686c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:22.200260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-caad82fd-51e6-4a48-afb9-5a00c12a9289 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:22.208381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-caad82fd-51e6-4a48-afb9-5a00c12a9289 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.155369 Sep 08 14:03:22.208948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 844/1583] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:22 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:22.252301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0ba72f6d-87af-4c1d-8bdd-44983087686c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:22.259974 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0ba72f6d-87af-4c1d-8bdd-44983087686c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.084197 Sep 08 14:03:22.260538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 740/1584] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:22 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:22.592080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b9a5d588-9a99-49d1-98a2-564785cdebfc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:22.594283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b9a5d588-9a99-49d1-98a2-564785cdebfc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:22.601344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b9a5d588-9a99-49d1-98a2-564785cdebfc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:22.601824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b9a5d588-9a99-49d1-98a2-564785cdebfc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:22.602301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b9a5d588-9a99-49d1-98a2-564785cdebfc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:22.603000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b9a5d588-9a99-49d1-98a2-564785cdebfc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:22.603453 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b9a5d588-9a99-49d1-98a2-564785cdebfc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:22.603880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b9a5d588-9a99-49d1-98a2-564785cdebfc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:22.634930 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b9a5d588-9a99-49d1-98a2-564785cdebfc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046801 Sep 08 14:03:22.635501 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 845/1585] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:22 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:22.752917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8fc1f854-924e-4689-9dfc-ad6513e7b9fa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:22.755115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8fc1f854-924e-4689-9dfc-ad6513e7b9fa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:22.761088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fc1f854-924e-4689-9dfc-ad6513e7b9fa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:22.761564 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fc1f854-924e-4689-9dfc-ad6513e7b9fa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:22.761988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fc1f854-924e-4689-9dfc-ad6513e7b9fa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:22.762704 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fc1f854-924e-4689-9dfc-ad6513e7b9fa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:22.763147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fc1f854-924e-4689-9dfc-ad6513e7b9fa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:22.763561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fc1f854-924e-4689-9dfc-ad6513e7b9fa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:22.794809 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8fc1f854-924e-4689-9dfc-ad6513e7b9fa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046280 Sep 08 14:03:22.795370 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 741/1586] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:22 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:23.014906 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-44b30c3b-46eb-469c-875f-4d4531d05e0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:23.017629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-44b30c3b-46eb-469c-875f-4d4531d05e0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:23.023693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44b30c3b-46eb-469c-875f-4d4531d05e0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:23.024179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44b30c3b-46eb-469c-875f-4d4531d05e0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:23.024624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44b30c3b-46eb-469c-875f-4d4531d05e0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:23.025333 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44b30c3b-46eb-469c-875f-4d4531d05e0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:23.025829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44b30c3b-46eb-469c-875f-4d4531d05e0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:23.026282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44b30c3b-46eb-469c-875f-4d4531d05e0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:23.056973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-44b30c3b-46eb-469c-875f-4d4531d05e0a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046118 Sep 08 14:03:23.057656 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 846/1587] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:23 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:23.232691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cd56eb1c-ec76-4ff6-a1da-0e7c95009c9c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:23.233299 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cd56eb1c-ec76-4ff6-a1da-0e7c95009c9c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:23.240568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd56eb1c-ec76-4ff6-a1da-0e7c95009c9c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:23.241057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd56eb1c-ec76-4ff6-a1da-0e7c95009c9c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:23.241490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd56eb1c-ec76-4ff6-a1da-0e7c95009c9c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:23.242212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd56eb1c-ec76-4ff6-a1da-0e7c95009c9c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:23.242666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd56eb1c-ec76-4ff6-a1da-0e7c95009c9c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:23.243088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd56eb1c-ec76-4ff6-a1da-0e7c95009c9c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:23.283101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-809518e6-b1c7-44b3-a29e-eba15f998f8d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:23.284164 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-809518e6-b1c7-44b3-a29e-eba15f998f8d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:23.290836 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-809518e6-b1c7-44b3-a29e-eba15f998f8d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:23.291359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-809518e6-b1c7-44b3-a29e-eba15f998f8d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:23.291802 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-809518e6-b1c7-44b3-a29e-eba15f998f8d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:23.292509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-809518e6-b1c7-44b3-a29e-eba15f998f8d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:23.292967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-809518e6-b1c7-44b3-a29e-eba15f998f8d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:23.293399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-809518e6-b1c7-44b3-a29e-eba15f998f8d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:23.357561 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-809518e6-b1c7-44b3-a29e-eba15f998f8d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:23.367039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-809518e6-b1c7-44b3-a29e-eba15f998f8d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.088052 Sep 08 14:03:23.367714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 847/1588] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:23 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:23.381439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-cd56eb1c-ec76-4ff6-a1da-0e7c95009c9c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:23.389376 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cd56eb1c-ec76-4ff6-a1da-0e7c95009c9c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.160591 Sep 08 14:03:23.389945 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 742/1589] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:23 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:23.647687 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9b0f0f50-6633-4723-939e-ec8239b780e2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:23.649876 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9b0f0f50-6633-4723-939e-ec8239b780e2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:23.656107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9b0f0f50-6633-4723-939e-ec8239b780e2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:23.656633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9b0f0f50-6633-4723-939e-ec8239b780e2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:23.657654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9b0f0f50-6633-4723-939e-ec8239b780e2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:23.658375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9b0f0f50-6633-4723-939e-ec8239b780e2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:23.658830 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9b0f0f50-6633-4723-939e-ec8239b780e2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:23.659262 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9b0f0f50-6633-4723-939e-ec8239b780e2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:23.690361 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9b0f0f50-6633-4723-939e-ec8239b780e2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046577 Sep 08 14:03:23.690929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 848/1590] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:23 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:23.808126 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1197ccec-a395-455f-8613-27b51c031031 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:23.810335 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1197ccec-a395-455f-8613-27b51c031031 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:23.817259 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1197ccec-a395-455f-8613-27b51c031031 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:23.817740 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1197ccec-a395-455f-8613-27b51c031031 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:23.818203 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1197ccec-a395-455f-8613-27b51c031031 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:23.818901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1197ccec-a395-455f-8613-27b51c031031 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:23.819350 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1197ccec-a395-455f-8613-27b51c031031 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:23.819767 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1197ccec-a395-455f-8613-27b51c031031 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:23.850089 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1197ccec-a395-455f-8613-27b51c031031 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045951 Sep 08 14:03:23.850696 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 743/1591] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:23 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:24.069654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ef2426d5-1555-49ba-872c-e694c011b1b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:24.071828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ef2426d5-1555-49ba-872c-e694c011b1b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:24.077762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef2426d5-1555-49ba-872c-e694c011b1b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:24.078284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef2426d5-1555-49ba-872c-e694c011b1b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:24.078730 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef2426d5-1555-49ba-872c-e694c011b1b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:24.079924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef2426d5-1555-49ba-872c-e694c011b1b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:24.080372 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef2426d5-1555-49ba-872c-e694c011b1b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:24.080790 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef2426d5-1555-49ba-872c-e694c011b1b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:24.112641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ef2426d5-1555-49ba-872c-e694c011b1b0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046924 Sep 08 14:03:24.113197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 849/1592] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:24 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:24.391121 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-38f8bf28-025e-4999-99ab-f5962a6586fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:24.392335 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-38f8bf28-025e-4999-99ab-f5962a6586fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:24.398859 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38f8bf28-025e-4999-99ab-f5962a6586fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:24.399340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38f8bf28-025e-4999-99ab-f5962a6586fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:24.399769 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38f8bf28-025e-4999-99ab-f5962a6586fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:24.400467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38f8bf28-025e-4999-99ab-f5962a6586fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:24.400912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38f8bf28-025e-4999-99ab-f5962a6586fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:24.401330 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-38f8bf28-025e-4999-99ab-f5962a6586fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:24.413067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f35e9ddc-8e90-498c-af27-439e7fd21004 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:24.413678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f35e9ddc-8e90-498c-af27-439e7fd21004 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:24.419988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f35e9ddc-8e90-498c-af27-439e7fd21004 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:24.420478 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f35e9ddc-8e90-498c-af27-439e7fd21004 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:24.420926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f35e9ddc-8e90-498c-af27-439e7fd21004 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:24.421639 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f35e9ddc-8e90-498c-af27-439e7fd21004 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:24.422113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f35e9ddc-8e90-498c-af27-439e7fd21004 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:24.422554 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f35e9ddc-8e90-498c-af27-439e7fd21004 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:24.474754 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-38f8bf28-025e-4999-99ab-f5962a6586fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:24.482835 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-38f8bf28-025e-4999-99ab-f5962a6586fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.095498 Sep 08 14:03:24.483403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 744/1593] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:24 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:24.561624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f35e9ddc-8e90-498c-af27-439e7fd21004 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:24.570423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f35e9ddc-8e90-498c-af27-439e7fd21004 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.161208 Sep 08 14:03:24.570988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 850/1594] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:24 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:24.704516 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2231cf0f-919e-4d41-b2b6-71c9231fc318 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:24.706811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2231cf0f-919e-4d41-b2b6-71c9231fc318 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:24.714277 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2231cf0f-919e-4d41-b2b6-71c9231fc318 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:24.714858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2231cf0f-919e-4d41-b2b6-71c9231fc318 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:24.715409 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2231cf0f-919e-4d41-b2b6-71c9231fc318 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:24.716406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2231cf0f-919e-4d41-b2b6-71c9231fc318 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:24.716906 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2231cf0f-919e-4d41-b2b6-71c9231fc318 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:24.717427 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2231cf0f-919e-4d41-b2b6-71c9231fc318 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:24.752673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2231cf0f-919e-4d41-b2b6-71c9231fc318 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053314 Sep 08 14:03:24.753343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 745/1595] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:24 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:24.863614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7c89e638-502d-444d-91fd-8e8d5f1a76fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:24.865778 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7c89e638-502d-444d-91fd-8e8d5f1a76fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:24.871777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7c89e638-502d-444d-91fd-8e8d5f1a76fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:24.872518 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7c89e638-502d-444d-91fd-8e8d5f1a76fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:24.872952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7c89e638-502d-444d-91fd-8e8d5f1a76fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:24.873634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7c89e638-502d-444d-91fd-8e8d5f1a76fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:24.874087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7c89e638-502d-444d-91fd-8e8d5f1a76fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:24.874538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7c89e638-502d-444d-91fd-8e8d5f1a76fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:24.904781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7c89e638-502d-444d-91fd-8e8d5f1a76fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045799 Sep 08 14:03:24.905342 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 851/1596] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:24 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:25.126281 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fae2d1ec-7798-4b1c-8be2-8bff03ec7feb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:25.128556 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fae2d1ec-7798-4b1c-8be2-8bff03ec7feb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:25.134582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fae2d1ec-7798-4b1c-8be2-8bff03ec7feb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:25.135065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fae2d1ec-7798-4b1c-8be2-8bff03ec7feb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:25.135498 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fae2d1ec-7798-4b1c-8be2-8bff03ec7feb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:25.136190 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fae2d1ec-7798-4b1c-8be2-8bff03ec7feb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:25.136921 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fae2d1ec-7798-4b1c-8be2-8bff03ec7feb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:25.137078 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fae2d1ec-7798-4b1c-8be2-8bff03ec7feb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:25.176460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fae2d1ec-7798-4b1c-8be2-8bff03ec7feb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054542 Sep 08 14:03:25.177334 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 746/1597] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:25 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:25.505429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b42eeac0-83dd-4a23-8e1c-d24debf70a04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:25.505976 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b42eeac0-83dd-4a23-8e1c-d24debf70a04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:25.512048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b42eeac0-83dd-4a23-8e1c-d24debf70a04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:25.512532 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b42eeac0-83dd-4a23-8e1c-d24debf70a04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:25.512973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b42eeac0-83dd-4a23-8e1c-d24debf70a04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:25.513667 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b42eeac0-83dd-4a23-8e1c-d24debf70a04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:25.514126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b42eeac0-83dd-4a23-8e1c-d24debf70a04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:25.514586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b42eeac0-83dd-4a23-8e1c-d24debf70a04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:25.576545 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b42eeac0-83dd-4a23-8e1c-d24debf70a04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:25.584828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b42eeac0-83dd-4a23-8e1c-d24debf70a04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.083377 Sep 08 14:03:25.585391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 852/1598] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:25 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:25.593524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8c31b459-5696-4fd7-b8a9-414bf687c651 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:25.594252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8c31b459-5696-4fd7-b8a9-414bf687c651 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:25.602063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8c31b459-5696-4fd7-b8a9-414bf687c651 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:25.602588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8c31b459-5696-4fd7-b8a9-414bf687c651 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:25.603026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8c31b459-5696-4fd7-b8a9-414bf687c651 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:25.604780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8c31b459-5696-4fd7-b8a9-414bf687c651 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:25.605235 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8c31b459-5696-4fd7-b8a9-414bf687c651 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:25.605661 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8c31b459-5696-4fd7-b8a9-414bf687c651 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:25.738743 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8c31b459-5696-4fd7-b8a9-414bf687c651 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:25.746531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8c31b459-5696-4fd7-b8a9-414bf687c651 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.156896 Sep 08 14:03:25.747355 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 747/1599] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:25 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:25.765884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6e6fdcfa-ac0d-409d-87ca-f1732a453167 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:25.768671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6e6fdcfa-ac0d-409d-87ca-f1732a453167 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:25.774707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6e6fdcfa-ac0d-409d-87ca-f1732a453167 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:25.775196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6e6fdcfa-ac0d-409d-87ca-f1732a453167 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:25.775643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6e6fdcfa-ac0d-409d-87ca-f1732a453167 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:25.776338 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6e6fdcfa-ac0d-409d-87ca-f1732a453167 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:25.776798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6e6fdcfa-ac0d-409d-87ca-f1732a453167 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:25.777229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6e6fdcfa-ac0d-409d-87ca-f1732a453167 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:25.810511 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6e6fdcfa-ac0d-409d-87ca-f1732a453167 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048563 Sep 08 14:03:25.811080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 853/1600] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:25 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:25.917302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-458a7d02-7356-4589-b2a2-afbd817b158a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:25.919601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-458a7d02-7356-4589-b2a2-afbd817b158a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:25.925949 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-458a7d02-7356-4589-b2a2-afbd817b158a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:25.926472 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-458a7d02-7356-4589-b2a2-afbd817b158a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:25.926913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-458a7d02-7356-4589-b2a2-afbd817b158a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:25.927614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-458a7d02-7356-4589-b2a2-afbd817b158a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:25.928070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-458a7d02-7356-4589-b2a2-afbd817b158a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:25.928498 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-458a7d02-7356-4589-b2a2-afbd817b158a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:25.958826 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-458a7d02-7356-4589-b2a2-afbd817b158a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045716 Sep 08 14:03:25.959417 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 748/1601] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:25 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:26.193479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d4daa8dd-f32f-4967-a762-3b40cd6e23c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:26.195676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d4daa8dd-f32f-4967-a762-3b40cd6e23c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:26.202047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4daa8dd-f32f-4967-a762-3b40cd6e23c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:26.202729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4daa8dd-f32f-4967-a762-3b40cd6e23c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:26.203204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4daa8dd-f32f-4967-a762-3b40cd6e23c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:26.203901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4daa8dd-f32f-4967-a762-3b40cd6e23c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:26.204345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4daa8dd-f32f-4967-a762-3b40cd6e23c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:26.204772 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4daa8dd-f32f-4967-a762-3b40cd6e23c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:26.234252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d4daa8dd-f32f-4967-a762-3b40cd6e23c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.044860 Sep 08 14:03:26.234820 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 854/1602] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:26 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:26.608339 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-daf06c0b-f817-4d62-9612-4506d427fbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:26.608940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-daf06c0b-f817-4d62-9612-4506d427fbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:26.616140 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-daf06c0b-f817-4d62-9612-4506d427fbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:26.616618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-daf06c0b-f817-4d62-9612-4506d427fbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:26.617049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-daf06c0b-f817-4d62-9612-4506d427fbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:26.617736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-daf06c0b-f817-4d62-9612-4506d427fbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:26.618209 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-daf06c0b-f817-4d62-9612-4506d427fbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:26.618632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-daf06c0b-f817-4d62-9612-4506d427fbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:26.682062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-daf06c0b-f817-4d62-9612-4506d427fbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:26.690321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-daf06c0b-f817-4d62-9612-4506d427fbed tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.085824 Sep 08 14:03:26.690908 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 749/1603] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:26 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:26.771791 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8f90d9ad-5981-439a-8f73-29420425f8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:26.772790 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8f90d9ad-5981-439a-8f73-29420425f8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:26.779278 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8f90d9ad-5981-439a-8f73-29420425f8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:26.779754 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8f90d9ad-5981-439a-8f73-29420425f8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:26.780181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8f90d9ad-5981-439a-8f73-29420425f8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:26.780863 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8f90d9ad-5981-439a-8f73-29420425f8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:26.781316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8f90d9ad-5981-439a-8f73-29420425f8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:26.782335 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8f90d9ad-5981-439a-8f73-29420425f8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:26.823499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-64b3cbfe-3c6b-4336-8a5d-a486e7ac968e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:26.825731 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-64b3cbfe-3c6b-4336-8a5d-a486e7ac968e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:26.831950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64b3cbfe-3c6b-4336-8a5d-a486e7ac968e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:26.832441 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64b3cbfe-3c6b-4336-8a5d-a486e7ac968e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:26.832877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64b3cbfe-3c6b-4336-8a5d-a486e7ac968e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:26.833569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64b3cbfe-3c6b-4336-8a5d-a486e7ac968e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:26.834018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64b3cbfe-3c6b-4336-8a5d-a486e7ac968e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:26.834470 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64b3cbfe-3c6b-4336-8a5d-a486e7ac968e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:26.866612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-64b3cbfe-3c6b-4336-8a5d-a486e7ac968e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047014 Sep 08 14:03:26.867252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 750/1604] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:26 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:26.917082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8f90d9ad-5981-439a-8f73-29420425f8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:26.925724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8f90d9ad-5981-439a-8f73-29420425f8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.158519 Sep 08 14:03:26.926296 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 855/1605] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:26 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:26.972108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e13e9def-df6d-46d2-8d37-51678e51c8ea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:26.974310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e13e9def-df6d-46d2-8d37-51678e51c8ea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:26.980288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e13e9def-df6d-46d2-8d37-51678e51c8ea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:26.980765 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e13e9def-df6d-46d2-8d37-51678e51c8ea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:26.981196 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e13e9def-df6d-46d2-8d37-51678e51c8ea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:26.981881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e13e9def-df6d-46d2-8d37-51678e51c8ea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:26.982407 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e13e9def-df6d-46d2-8d37-51678e51c8ea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:26.982833 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e13e9def-df6d-46d2-8d37-51678e51c8ea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:27.012796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e13e9def-df6d-46d2-8d37-51678e51c8ea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045373 Sep 08 14:03:27.013356 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 751/1606] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:26 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:27.248033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b018235c-b40f-4c86-bb0a-fd7c0e4f667d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:27.250153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b018235c-b40f-4c86-bb0a-fd7c0e4f667d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:27.257058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b018235c-b40f-4c86-bb0a-fd7c0e4f667d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:27.257536 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b018235c-b40f-4c86-bb0a-fd7c0e4f667d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:27.257970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b018235c-b40f-4c86-bb0a-fd7c0e4f667d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:27.258685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b018235c-b40f-4c86-bb0a-fd7c0e4f667d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:27.259139 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b018235c-b40f-4c86-bb0a-fd7c0e4f667d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:27.259594 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b018235c-b40f-4c86-bb0a-fd7c0e4f667d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:27.290238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b018235c-b40f-4c86-bb0a-fd7c0e4f667d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046498 Sep 08 14:03:27.290811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 856/1607] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:27 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:27.720534 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-860b83d6-75d1-4658-a8d0-625295f204b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:27.721137 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-860b83d6-75d1-4658-a8d0-625295f204b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:27.727542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-860b83d6-75d1-4658-a8d0-625295f204b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:27.728020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-860b83d6-75d1-4658-a8d0-625295f204b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:27.728449 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-860b83d6-75d1-4658-a8d0-625295f204b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:27.729129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-860b83d6-75d1-4658-a8d0-625295f204b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:27.729574 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-860b83d6-75d1-4658-a8d0-625295f204b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:27.729990 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-860b83d6-75d1-4658-a8d0-625295f204b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:27.792383 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-860b83d6-75d1-4658-a8d0-625295f204b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:27.800521 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-860b83d6-75d1-4658-a8d0-625295f204b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.084314 Sep 08 14:03:27.801077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 752/1608] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:27 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:27.879708 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-450ea043-2f34-4a81-9b92-560131689709 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:27.881852 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-450ea043-2f34-4a81-9b92-560131689709 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:27.887831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-450ea043-2f34-4a81-9b92-560131689709 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:27.888301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-450ea043-2f34-4a81-9b92-560131689709 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:27.888728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-450ea043-2f34-4a81-9b92-560131689709 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:27.889406 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-450ea043-2f34-4a81-9b92-560131689709 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:27.889905 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-450ea043-2f34-4a81-9b92-560131689709 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:27.890318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-450ea043-2f34-4a81-9b92-560131689709 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:27.920539 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-450ea043-2f34-4a81-9b92-560131689709 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045334 Sep 08 14:03:27.921093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 857/1609] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:27 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:27.955156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-decfb818-288d-4311-9f5b-95b8c1ae0711 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:27.955789 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-decfb818-288d-4311-9f5b-95b8c1ae0711 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:27.961797 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-decfb818-288d-4311-9f5b-95b8c1ae0711 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:27.962320 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-decfb818-288d-4311-9f5b-95b8c1ae0711 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:27.962754 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-decfb818-288d-4311-9f5b-95b8c1ae0711 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:27.963446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-decfb818-288d-4311-9f5b-95b8c1ae0711 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:27.963890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-decfb818-288d-4311-9f5b-95b8c1ae0711 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:27.964309 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-decfb818-288d-4311-9f5b-95b8c1ae0711 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:28.027268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-788c6d88-72ac-4666-9729-d4ad64b25576 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:28.030609 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-788c6d88-72ac-4666-9729-d4ad64b25576 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:28.038795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-788c6d88-72ac-4666-9729-d4ad64b25576 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:28.039632 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-788c6d88-72ac-4666-9729-d4ad64b25576 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:28.040224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-788c6d88-72ac-4666-9729-d4ad64b25576 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:28.040966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-788c6d88-72ac-4666-9729-d4ad64b25576 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:28.041402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-788c6d88-72ac-4666-9729-d4ad64b25576 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:28.041835 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-788c6d88-72ac-4666-9729-d4ad64b25576 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:28.074948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-788c6d88-72ac-4666-9729-d4ad64b25576 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052835 Sep 08 14:03:28.075526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 858/1610] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:28 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:28.103462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-decfb818-288d-4311-9f5b-95b8c1ae0711 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:28.112022 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-decfb818-288d-4311-9f5b-95b8c1ae0711 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.160755 Sep 08 14:03:28.112615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 753/1611] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:27 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:28.303832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-26174c97-9000-4814-93ea-915706d7cffa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:28.305908 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-26174c97-9000-4814-93ea-915706d7cffa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:28.312466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26174c97-9000-4814-93ea-915706d7cffa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:28.312950 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26174c97-9000-4814-93ea-915706d7cffa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:28.313387 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26174c97-9000-4814-93ea-915706d7cffa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:28.314087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26174c97-9000-4814-93ea-915706d7cffa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:28.314571 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26174c97-9000-4814-93ea-915706d7cffa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:28.315000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26174c97-9000-4814-93ea-915706d7cffa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:28.328724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:28.345888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-26174c97-9000-4814-93ea-915706d7cffa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046240 Sep 08 14:03:28.346491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 859/1612] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:28 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:28.824237 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4d2244ce-c973-40a1-917c-add60b574465 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:28.824904 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4d2244ce-c973-40a1-917c-add60b574465 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:28.831702 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d2244ce-c973-40a1-917c-add60b574465 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:28.832187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d2244ce-c973-40a1-917c-add60b574465 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:28.832620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d2244ce-c973-40a1-917c-add60b574465 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:28.833314 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d2244ce-c973-40a1-917c-add60b574465 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:28.833764 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d2244ce-c973-40a1-917c-add60b574465 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:28.834209 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d2244ce-c973-40a1-917c-add60b574465 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:28.896962 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4d2244ce-c973-40a1-917c-add60b574465 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:28.904642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4d2244ce-c973-40a1-917c-add60b574465 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.084314 Sep 08 14:03:28.905402 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 754/1613] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:28 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:28.933789 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f8b38aa0-553d-4c08-8d7e-9fcedc98bec3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:28.935988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f8b38aa0-553d-4c08-8d7e-9fcedc98bec3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:28.942580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8b38aa0-553d-4c08-8d7e-9fcedc98bec3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:28.943062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8b38aa0-553d-4c08-8d7e-9fcedc98bec3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:28.943495 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8b38aa0-553d-4c08-8d7e-9fcedc98bec3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:28.944184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8b38aa0-553d-4c08-8d7e-9fcedc98bec3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:28.944634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8b38aa0-553d-4c08-8d7e-9fcedc98bec3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:28.945053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8b38aa0-553d-4c08-8d7e-9fcedc98bec3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:28.975553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f8b38aa0-553d-4c08-8d7e-9fcedc98bec3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045893 Sep 08 14:03:28.976114 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 860/1614] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:28 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:29.087812 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1d9ef020-8f89-42ae-a537-f3752682bd79 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:29.090042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1d9ef020-8f89-42ae-a537-f3752682bd79 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:29.097305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d9ef020-8f89-42ae-a537-f3752682bd79 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:29.097789 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d9ef020-8f89-42ae-a537-f3752682bd79 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:29.098295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d9ef020-8f89-42ae-a537-f3752682bd79 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:29.104183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d9ef020-8f89-42ae-a537-f3752682bd79 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:29.104183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d9ef020-8f89-42ae-a537-f3752682bd79 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:29.104183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d9ef020-8f89-42ae-a537-f3752682bd79 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:29.118652 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:29.132125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1d9ef020-8f89-42ae-a537-f3752682bd79 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048243 Sep 08 14:03:29.132720 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 755/1615] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:29 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:29.136089 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c7578c40-6e40-456b-abcf-4179cdcf65ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:29.136733 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c7578c40-6e40-456b-abcf-4179cdcf65ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:29.142802 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c7578c40-6e40-456b-abcf-4179cdcf65ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:29.143286 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c7578c40-6e40-456b-abcf-4179cdcf65ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:29.143722 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c7578c40-6e40-456b-abcf-4179cdcf65ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:29.144465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c7578c40-6e40-456b-abcf-4179cdcf65ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:29.144862 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c7578c40-6e40-456b-abcf-4179cdcf65ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:29.145314 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c7578c40-6e40-456b-abcf-4179cdcf65ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:29.289462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c7578c40-6e40-456b-abcf-4179cdcf65ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:29.297305 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c7578c40-6e40-456b-abcf-4179cdcf65ba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.165141 Sep 08 14:03:29.297865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 861/1616] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:29 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:29.358983 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7a1143a0-9a36-4bf1-b8c5-4b9c705d222b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:29.361428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7a1143a0-9a36-4bf1-b8c5-4b9c705d222b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:29.367903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a1143a0-9a36-4bf1-b8c5-4b9c705d222b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:29.368378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a1143a0-9a36-4bf1-b8c5-4b9c705d222b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:29.368802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a1143a0-9a36-4bf1-b8c5-4b9c705d222b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:29.369486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a1143a0-9a36-4bf1-b8c5-4b9c705d222b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:29.369930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a1143a0-9a36-4bf1-b8c5-4b9c705d222b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:29.370372 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a1143a0-9a36-4bf1-b8c5-4b9c705d222b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:29.402159 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7a1143a0-9a36-4bf1-b8c5-4b9c705d222b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047148 Sep 08 14:03:29.402630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 756/1617] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:29 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:29.938933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e99bd030-1432-4a35-b107-ba67beff32f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:29.938933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e99bd030-1432-4a35-b107-ba67beff32f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:29.938933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e99bd030-1432-4a35-b107-ba67beff32f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:29.938933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e99bd030-1432-4a35-b107-ba67beff32f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:29.940272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e99bd030-1432-4a35-b107-ba67beff32f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:29.940272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e99bd030-1432-4a35-b107-ba67beff32f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:29.940618 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e99bd030-1432-4a35-b107-ba67beff32f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:29.941052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e99bd030-1432-4a35-b107-ba67beff32f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:29.988624 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1fc369e3-3990-4391-aec0-8dbae86df482 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:29.990811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1fc369e3-3990-4391-aec0-8dbae86df482 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:29.998180 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1fc369e3-3990-4391-aec0-8dbae86df482 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:29.998782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1fc369e3-3990-4391-aec0-8dbae86df482 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:29.999180 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1fc369e3-3990-4391-aec0-8dbae86df482 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:29.999892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1fc369e3-3990-4391-aec0-8dbae86df482 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:30.000955 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1fc369e3-3990-4391-aec0-8dbae86df482 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:30.001380 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1fc369e3-3990-4391-aec0-8dbae86df482 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:30.034255 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1fc369e3-3990-4391-aec0-8dbae86df482 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050007 Sep 08 14:03:30.034838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 757/1618] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:29 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:30.066041 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e99bd030-1432-4a35-b107-ba67beff32f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:30.073670 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e99bd030-1432-4a35-b107-ba67beff32f4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.147556 Sep 08 14:03:30.074247 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 862/1619] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:29 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:30.143890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f49bca68-ad42-4bf3-b725-574e92eaac61 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:30.146064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f49bca68-ad42-4bf3-b725-574e92eaac61 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:30.152063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f49bca68-ad42-4bf3-b725-574e92eaac61 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:30.152546 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f49bca68-ad42-4bf3-b725-574e92eaac61 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:30.152983 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f49bca68-ad42-4bf3-b725-574e92eaac61 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:30.153676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f49bca68-ad42-4bf3-b725-574e92eaac61 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:30.154128 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f49bca68-ad42-4bf3-b725-574e92eaac61 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:30.154604 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f49bca68-ad42-4bf3-b725-574e92eaac61 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:30.185064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f49bca68-ad42-4bf3-b725-574e92eaac61 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045074 Sep 08 14:03:30.185542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 758/1620] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:30 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:30.321876 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6cc9667a-1b45-4d2d-895f-e8444dd244f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:30.322567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6cc9667a-1b45-4d2d-895f-e8444dd244f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:30.329923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6cc9667a-1b45-4d2d-895f-e8444dd244f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:30.330651 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6cc9667a-1b45-4d2d-895f-e8444dd244f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:30.331075 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6cc9667a-1b45-4d2d-895f-e8444dd244f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:30.331786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6cc9667a-1b45-4d2d-895f-e8444dd244f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:30.332325 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6cc9667a-1b45-4d2d-895f-e8444dd244f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:30.332935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6cc9667a-1b45-4d2d-895f-e8444dd244f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:30.419306 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b1e4a45b-3c9a-4dcf-ac7f-f1609b2ecf04 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:30.424480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b1e4a45b-3c9a-4dcf-ac7f-f1609b2ecf04 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:30.436088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1e4a45b-3c9a-4dcf-ac7f-f1609b2ecf04 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:30.436592 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1e4a45b-3c9a-4dcf-ac7f-f1609b2ecf04 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:30.437373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1e4a45b-3c9a-4dcf-ac7f-f1609b2ecf04 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:30.438224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1e4a45b-3c9a-4dcf-ac7f-f1609b2ecf04 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:30.438744 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1e4a45b-3c9a-4dcf-ac7f-f1609b2ecf04 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:30.439231 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1e4a45b-3c9a-4dcf-ac7f-f1609b2ecf04 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:30.478407 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b1e4a45b-3c9a-4dcf-ac7f-f1609b2ecf04 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.063186 Sep 08 14:03:30.478991 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 759/1621] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:30 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:30.495343 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6cc9667a-1b45-4d2d-895f-e8444dd244f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:30.504567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6cc9667a-1b45-4d2d-895f-e8444dd244f9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.187111 Sep 08 14:03:30.508605 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 863/1622] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:30 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:31.047348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c18f2075-dc6b-40df-a001-f59938f29057 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:31.052179 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c18f2075-dc6b-40df-a001-f59938f29057 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:31.059587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c18f2075-dc6b-40df-a001-f59938f29057 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:31.060181 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c18f2075-dc6b-40df-a001-f59938f29057 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:31.060696 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c18f2075-dc6b-40df-a001-f59938f29057 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:31.061507 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c18f2075-dc6b-40df-a001-f59938f29057 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:31.061972 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c18f2075-dc6b-40df-a001-f59938f29057 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:31.062429 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c18f2075-dc6b-40df-a001-f59938f29057 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:31.101599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-60d53815-1278-4071-bc6e-84e1ea9af161 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:31.102215 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c18f2075-dc6b-40df-a001-f59938f29057 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054589 Sep 08 14:03:31.102314 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-60d53815-1278-4071-bc6e-84e1ea9af161 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:31.103252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 760/1623] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:31 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:31.105697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-60d53815-1278-4071-bc6e-84e1ea9af161 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:31.106205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-60d53815-1278-4071-bc6e-84e1ea9af161 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:31.106635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-60d53815-1278-4071-bc6e-84e1ea9af161 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:31.107337 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-60d53815-1278-4071-bc6e-84e1ea9af161 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:31.107782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-60d53815-1278-4071-bc6e-84e1ea9af161 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:31.108204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-60d53815-1278-4071-bc6e-84e1ea9af161 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:31.191108 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-60d53815-1278-4071-bc6e-84e1ea9af161 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:31.199404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-994d3b8a-0b33-4f68-aa2a-4cc07fc60f01 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:31.201543 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-994d3b8a-0b33-4f68-aa2a-4cc07fc60f01 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:31.203263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-60d53815-1278-4071-bc6e-84e1ea9af161 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.110044 Sep 08 14:03:31.207909 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 864/1624] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:31 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:31.208662 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-994d3b8a-0b33-4f68-aa2a-4cc07fc60f01 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:31.209219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-994d3b8a-0b33-4f68-aa2a-4cc07fc60f01 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:31.209671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-994d3b8a-0b33-4f68-aa2a-4cc07fc60f01 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:31.212446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-994d3b8a-0b33-4f68-aa2a-4cc07fc60f01 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:31.212446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-994d3b8a-0b33-4f68-aa2a-4cc07fc60f01 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:31.212705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-994d3b8a-0b33-4f68-aa2a-4cc07fc60f01 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:31.253872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-994d3b8a-0b33-4f68-aa2a-4cc07fc60f01 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058454 Sep 08 14:03:31.254481 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 761/1625] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:31 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:31.492103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b8e888fb-f361-4f37-8ae3-aa43baf3ea95 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:31.494201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b8e888fb-f361-4f37-8ae3-aa43baf3ea95 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:31.501133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b8e888fb-f361-4f37-8ae3-aa43baf3ea95 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:31.501794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b8e888fb-f361-4f37-8ae3-aa43baf3ea95 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:31.502747 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b8e888fb-f361-4f37-8ae3-aa43baf3ea95 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:31.503821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b8e888fb-f361-4f37-8ae3-aa43baf3ea95 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:31.504622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b8e888fb-f361-4f37-8ae3-aa43baf3ea95 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:31.505156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b8e888fb-f361-4f37-8ae3-aa43baf3ea95 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:31.529737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a3eab323-e00f-4860-b0b5-969e93d14d72 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:31.530468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a3eab323-e00f-4860-b0b5-969e93d14d72 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:31.537756 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a3eab323-e00f-4860-b0b5-969e93d14d72 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:31.538276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a3eab323-e00f-4860-b0b5-969e93d14d72 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:31.538713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a3eab323-e00f-4860-b0b5-969e93d14d72 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:31.539358 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b8e888fb-f361-4f37-8ae3-aa43baf3ea95 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051355 Sep 08 14:03:31.539562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a3eab323-e00f-4860-b0b5-969e93d14d72 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:31.539853 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a3eab323-e00f-4860-b0b5-969e93d14d72 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:31.539965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 865/1626] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:31 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:31.540273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a3eab323-e00f-4860-b0b5-969e93d14d72 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:31.766520 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a3eab323-e00f-4860-b0b5-969e93d14d72 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:31.776045 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a3eab323-e00f-4860-b0b5-969e93d14d72 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.250137 Sep 08 14:03:31.777127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 762/1627] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:31 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:32.109753 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9b65b3be-c380-42c8-9229-2c590b697e53 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:32.111965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9b65b3be-c380-42c8-9229-2c590b697e53 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:32.118571 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9b65b3be-c380-42c8-9229-2c590b697e53 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:32.119055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9b65b3be-c380-42c8-9229-2c590b697e53 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:32.119493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9b65b3be-c380-42c8-9229-2c590b697e53 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:32.120189 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9b65b3be-c380-42c8-9229-2c590b697e53 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:32.120642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9b65b3be-c380-42c8-9229-2c590b697e53 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:32.121072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9b65b3be-c380-42c8-9229-2c590b697e53 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:32.152965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9b65b3be-c380-42c8-9229-2c590b697e53 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047195 Sep 08 14:03:32.153535 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 866/1628] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:32 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:32.237411 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-32a089f5-85c2-412c-9ce4-29db5bdedee7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:32.238007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-32a089f5-85c2-412c-9ce4-29db5bdedee7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:32.244616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-32a089f5-85c2-412c-9ce4-29db5bdedee7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:32.245087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-32a089f5-85c2-412c-9ce4-29db5bdedee7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:32.245516 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-32a089f5-85c2-412c-9ce4-29db5bdedee7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:32.246227 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-32a089f5-85c2-412c-9ce4-29db5bdedee7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:32.246935 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-32a089f5-85c2-412c-9ce4-29db5bdedee7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:32.247353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-32a089f5-85c2-412c-9ce4-29db5bdedee7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:32.268416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8fdc1b6d-5877-48ee-aaba-2db21580f2fa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:32.270793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8fdc1b6d-5877-48ee-aaba-2db21580f2fa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:32.276676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fdc1b6d-5877-48ee-aaba-2db21580f2fa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:32.277158 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fdc1b6d-5877-48ee-aaba-2db21580f2fa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:32.277597 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fdc1b6d-5877-48ee-aaba-2db21580f2fa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:32.278327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fdc1b6d-5877-48ee-aaba-2db21580f2fa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:32.278952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fdc1b6d-5877-48ee-aaba-2db21580f2fa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:32.279452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fdc1b6d-5877-48ee-aaba-2db21580f2fa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:32.310884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8fdc1b6d-5877-48ee-aaba-2db21580f2fa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046880 Sep 08 14:03:32.311543 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 867/1629] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:32 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:32.320001 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-32a089f5-85c2-412c-9ce4-29db5bdedee7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:32.327988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-32a089f5-85c2-412c-9ce4-29db5bdedee7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.094961 Sep 08 14:03:32.328520 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 763/1630] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:32 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:32.553677 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-41473ff1-bf98-4ab5-a9e7-f8e3179c2a7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:32.555785 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-41473ff1-bf98-4ab5-a9e7-f8e3179c2a7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:32.561761 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-41473ff1-bf98-4ab5-a9e7-f8e3179c2a7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:32.562327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-41473ff1-bf98-4ab5-a9e7-f8e3179c2a7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:32.563413 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-41473ff1-bf98-4ab5-a9e7-f8e3179c2a7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:32.564124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-41473ff1-bf98-4ab5-a9e7-f8e3179c2a7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:32.564578 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-41473ff1-bf98-4ab5-a9e7-f8e3179c2a7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:32.565006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-41473ff1-bf98-4ab5-a9e7-f8e3179c2a7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:32.596173 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-41473ff1-bf98-4ab5-a9e7-f8e3179c2a7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046867 Sep 08 14:03:32.596768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 868/1631] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:32 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:32.801660 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-160b2454-0393-4be7-a665-572b7eb73628 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:32.802540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-160b2454-0393-4be7-a665-572b7eb73628 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:32.810183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-160b2454-0393-4be7-a665-572b7eb73628 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:32.810605 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-160b2454-0393-4be7-a665-572b7eb73628 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:32.811035 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-160b2454-0393-4be7-a665-572b7eb73628 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:32.811716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-160b2454-0393-4be7-a665-572b7eb73628 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:32.812162 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-160b2454-0393-4be7-a665-572b7eb73628 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:32.812586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-160b2454-0393-4be7-a665-572b7eb73628 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:32.947325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-160b2454-0393-4be7-a665-572b7eb73628 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:32.955954 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-160b2454-0393-4be7-a665-572b7eb73628 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.158843 Sep 08 14:03:32.956541 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 764/1632] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:32 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:33.165148 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-38c52196-9f30-4df0-933a-123247c1fe74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:33.167301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-38c52196-9f30-4df0-933a-123247c1fe74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:33.174182 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38c52196-9f30-4df0-933a-123247c1fe74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:33.174671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38c52196-9f30-4df0-933a-123247c1fe74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:33.175104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38c52196-9f30-4df0-933a-123247c1fe74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:33.175795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38c52196-9f30-4df0-933a-123247c1fe74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:33.176243 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38c52196-9f30-4df0-933a-123247c1fe74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:33.176660 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38c52196-9f30-4df0-933a-123247c1fe74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:33.207737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-38c52196-9f30-4df0-933a-123247c1fe74 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046513 Sep 08 14:03:33.208526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 869/1633] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:33 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:33.326249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3a661be4-b22b-4800-a2c3-b96650edd664 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:33.329027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3a661be4-b22b-4800-a2c3-b96650edd664 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:33.335512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3a661be4-b22b-4800-a2c3-b96650edd664 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:33.335992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3a661be4-b22b-4800-a2c3-b96650edd664 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:33.336422 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3a661be4-b22b-4800-a2c3-b96650edd664 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:33.337116 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3a661be4-b22b-4800-a2c3-b96650edd664 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:33.337565 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3a661be4-b22b-4800-a2c3-b96650edd664 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:33.337981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3a661be4-b22b-4800-a2c3-b96650edd664 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:33.351319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-46404989-156e-4705-90d8-440583967fca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:33.351937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-46404989-156e-4705-90d8-440583967fca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:33.358049 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-46404989-156e-4705-90d8-440583967fca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:33.358588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-46404989-156e-4705-90d8-440583967fca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:33.359028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-46404989-156e-4705-90d8-440583967fca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:33.359733 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-46404989-156e-4705-90d8-440583967fca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:33.360206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-46404989-156e-4705-90d8-440583967fca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:33.360627 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-46404989-156e-4705-90d8-440583967fca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:33.369467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3a661be4-b22b-4800-a2c3-b96650edd664 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047310 Sep 08 14:03:33.370404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 765/1634] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:33 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:33.432894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-46404989-156e-4705-90d8-440583967fca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:33.442761 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-46404989-156e-4705-90d8-440583967fca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.095409 Sep 08 14:03:33.443381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 870/1635] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:33 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:33.608690 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e1d26518-a70a-4663-8de0-b13fbc97e724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:33.610880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e1d26518-a70a-4663-8de0-b13fbc97e724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:33.612730 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:33.617519 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1d26518-a70a-4663-8de0-b13fbc97e724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:33.617999 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1d26518-a70a-4663-8de0-b13fbc97e724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:33.618461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1d26518-a70a-4663-8de0-b13fbc97e724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:33.619152 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1d26518-a70a-4663-8de0-b13fbc97e724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:33.619594 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1d26518-a70a-4663-8de0-b13fbc97e724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:33.620010 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1d26518-a70a-4663-8de0-b13fbc97e724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:33.652730 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e1d26518-a70a-4663-8de0-b13fbc97e724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048118 Sep 08 14:03:33.653289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 766/1636] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:33 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:33.979528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c6af9825-20d7-4cc4-a4cb-b3093bd5e56b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:33.980132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c6af9825-20d7-4cc4-a4cb-b3093bd5e56b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:33.986692 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c6af9825-20d7-4cc4-a4cb-b3093bd5e56b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:33.987178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c6af9825-20d7-4cc4-a4cb-b3093bd5e56b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:33.987616 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c6af9825-20d7-4cc4-a4cb-b3093bd5e56b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:33.988572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c6af9825-20d7-4cc4-a4cb-b3093bd5e56b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:33.988826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c6af9825-20d7-4cc4-a4cb-b3093bd5e56b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:33.989251 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c6af9825-20d7-4cc4-a4cb-b3093bd5e56b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:34.134867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c6af9825-20d7-4cc4-a4cb-b3093bd5e56b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:34.143318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c6af9825-20d7-4cc4-a4cb-b3093bd5e56b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.167618 Sep 08 14:03:34.143887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 871/1637] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:33 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:34.219685 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-339ffb42-6b0d-4a65-935c-9e6dd21a9833 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:34.221784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-339ffb42-6b0d-4a65-935c-9e6dd21a9833 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:34.227801 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-339ffb42-6b0d-4a65-935c-9e6dd21a9833 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:34.228289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-339ffb42-6b0d-4a65-935c-9e6dd21a9833 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:34.228728 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-339ffb42-6b0d-4a65-935c-9e6dd21a9833 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:34.229422 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-339ffb42-6b0d-4a65-935c-9e6dd21a9833 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:34.229877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-339ffb42-6b0d-4a65-935c-9e6dd21a9833 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:34.230338 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-339ffb42-6b0d-4a65-935c-9e6dd21a9833 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:34.260317 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-339ffb42-6b0d-4a65-935c-9e6dd21a9833 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044722 Sep 08 14:03:34.260884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 767/1638] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:34 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:34.384913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5a0dc977-ae9c-4c0c-b35a-a271cfdb1bf0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:34.387114 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5a0dc977-ae9c-4c0c-b35a-a271cfdb1bf0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:34.394664 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a0dc977-ae9c-4c0c-b35a-a271cfdb1bf0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:34.395159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a0dc977-ae9c-4c0c-b35a-a271cfdb1bf0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:34.395628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a0dc977-ae9c-4c0c-b35a-a271cfdb1bf0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:34.396334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a0dc977-ae9c-4c0c-b35a-a271cfdb1bf0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:34.396795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a0dc977-ae9c-4c0c-b35a-a271cfdb1bf0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:34.397216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a0dc977-ae9c-4c0c-b35a-a271cfdb1bf0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:34.427381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5a0dc977-ae9c-4c0c-b35a-a271cfdb1bf0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047248 Sep 08 14:03:34.427954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 872/1639] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:34 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:34.466548 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3aa7a949-a3b7-4b5f-816f-3d93536c9528 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:34.467162 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3aa7a949-a3b7-4b5f-816f-3d93536c9528 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:34.473371 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3aa7a949-a3b7-4b5f-816f-3d93536c9528 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:34.473793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3aa7a949-a3b7-4b5f-816f-3d93536c9528 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:34.474259 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3aa7a949-a3b7-4b5f-816f-3d93536c9528 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:34.474998 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3aa7a949-a3b7-4b5f-816f-3d93536c9528 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:34.475452 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3aa7a949-a3b7-4b5f-816f-3d93536c9528 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:34.475876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3aa7a949-a3b7-4b5f-816f-3d93536c9528 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:34.539316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3aa7a949-a3b7-4b5f-816f-3d93536c9528 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:34.547711 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3aa7a949-a3b7-4b5f-816f-3d93536c9528 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.085224 Sep 08 14:03:34.548322 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 768/1640] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:34 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:34.665151 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5996865e-614d-4555-a946-2bfd2707f0d0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:34.667352 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5996865e-614d-4555-a946-2bfd2707f0d0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:34.674498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5996865e-614d-4555-a946-2bfd2707f0d0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:34.675538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5996865e-614d-4555-a946-2bfd2707f0d0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:34.675538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5996865e-614d-4555-a946-2bfd2707f0d0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:34.676111 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5996865e-614d-4555-a946-2bfd2707f0d0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:34.676571 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5996865e-614d-4555-a946-2bfd2707f0d0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:34.676999 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5996865e-614d-4555-a946-2bfd2707f0d0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:34.707118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5996865e-614d-4555-a946-2bfd2707f0d0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045981 Sep 08 14:03:34.707678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 873/1641] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:34 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:35.166670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8b802a80-c278-4a18-8da5-cba2a67dc31c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:35.167280 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8b802a80-c278-4a18-8da5-cba2a67dc31c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:35.174565 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8b802a80-c278-4a18-8da5-cba2a67dc31c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:35.175068 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8b802a80-c278-4a18-8da5-cba2a67dc31c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:35.175538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8b802a80-c278-4a18-8da5-cba2a67dc31c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:35.176238 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8b802a80-c278-4a18-8da5-cba2a67dc31c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:35.176692 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8b802a80-c278-4a18-8da5-cba2a67dc31c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:35.177113 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8b802a80-c278-4a18-8da5-cba2a67dc31c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:35.278439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a152d7cc-5199-4140-b61d-868658367715 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:35.280859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a152d7cc-5199-4140-b61d-868658367715 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:35.288214 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a152d7cc-5199-4140-b61d-868658367715 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:35.288712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a152d7cc-5199-4140-b61d-868658367715 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:35.289183 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a152d7cc-5199-4140-b61d-868658367715 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:35.289892 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a152d7cc-5199-4140-b61d-868658367715 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:35.290390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a152d7cc-5199-4140-b61d-868658367715 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:35.290827 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a152d7cc-5199-4140-b61d-868658367715 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:35.321010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a152d7cc-5199-4140-b61d-868658367715 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050035 Sep 08 14:03:35.321588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 874/1642] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:35 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:35.322032 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8b802a80-c278-4a18-8da5-cba2a67dc31c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:35.330781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8b802a80-c278-4a18-8da5-cba2a67dc31c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.167955 Sep 08 14:03:35.331357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 769/1643] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:35 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:35.440212 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-39efb718-f400-47f0-b03b-848378fc5fa2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:35.442302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-39efb718-f400-47f0-b03b-848378fc5fa2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:35.448678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-39efb718-f400-47f0-b03b-848378fc5fa2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:35.449156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-39efb718-f400-47f0-b03b-848378fc5fa2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:35.449587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-39efb718-f400-47f0-b03b-848378fc5fa2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:35.450303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-39efb718-f400-47f0-b03b-848378fc5fa2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:35.450749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-39efb718-f400-47f0-b03b-848378fc5fa2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:35.451170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-39efb718-f400-47f0-b03b-848378fc5fa2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:35.480818 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-39efb718-f400-47f0-b03b-848378fc5fa2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044665 Sep 08 14:03:35.481364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 875/1644] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:35 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:35.571228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f24b096f-6190-4645-9450-20fbd993026f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:35.571860 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f24b096f-6190-4645-9450-20fbd993026f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:35.578936 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f24b096f-6190-4645-9450-20fbd993026f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:35.579443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f24b096f-6190-4645-9450-20fbd993026f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:35.579833 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f24b096f-6190-4645-9450-20fbd993026f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:35.580516 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f24b096f-6190-4645-9450-20fbd993026f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:35.580958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f24b096f-6190-4645-9450-20fbd993026f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:35.581373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f24b096f-6190-4645-9450-20fbd993026f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:35.644892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f24b096f-6190-4645-9450-20fbd993026f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:35.652636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f24b096f-6190-4645-9450-20fbd993026f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.085230 Sep 08 14:03:35.653197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 770/1645] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:35 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:35.719738 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ac3f7471-831f-44bf-bd8d-e504a67cce5b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:35.721796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ac3f7471-831f-44bf-bd8d-e504a67cce5b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:35.727887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ac3f7471-831f-44bf-bd8d-e504a67cce5b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:35.728369 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ac3f7471-831f-44bf-bd8d-e504a67cce5b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:35.728809 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ac3f7471-831f-44bf-bd8d-e504a67cce5b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:35.729499 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ac3f7471-831f-44bf-bd8d-e504a67cce5b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:35.729952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ac3f7471-831f-44bf-bd8d-e504a67cce5b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:35.730407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ac3f7471-831f-44bf-bd8d-e504a67cce5b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:35.732780 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:35.761833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ac3f7471-831f-44bf-bd8d-e504a67cce5b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046316 Sep 08 14:03:35.762417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 876/1646] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:35 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:36.334313 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2824f62c-fa95-48ac-8a3b-bd141bdc6e00 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:36.336520 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2824f62c-fa95-48ac-8a3b-bd141bdc6e00 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:36.343075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2824f62c-fa95-48ac-8a3b-bd141bdc6e00 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:36.343565 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2824f62c-fa95-48ac-8a3b-bd141bdc6e00 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:36.343997 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2824f62c-fa95-48ac-8a3b-bd141bdc6e00 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:36.344687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2824f62c-fa95-48ac-8a3b-bd141bdc6e00 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:36.345140 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2824f62c-fa95-48ac-8a3b-bd141bdc6e00 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:36.345559 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2824f62c-fa95-48ac-8a3b-bd141bdc6e00 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:36.354524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2f1dd669-684f-423c-84ac-53298ab95cfd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:36.355140 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2f1dd669-684f-423c-84ac-53298ab95cfd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:36.361284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f1dd669-684f-423c-84ac-53298ab95cfd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:36.361772 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f1dd669-684f-423c-84ac-53298ab95cfd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:36.362273 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f1dd669-684f-423c-84ac-53298ab95cfd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:36.362946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f1dd669-684f-423c-84ac-53298ab95cfd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:36.363410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f1dd669-684f-423c-84ac-53298ab95cfd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:36.363839 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f1dd669-684f-423c-84ac-53298ab95cfd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:36.377076 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2824f62c-fa95-48ac-8a3b-bd141bdc6e00 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046893 Sep 08 14:03:36.377648 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 771/1647] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:36 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:36.493328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1d2a6ff0-bef9-4728-a762-5485a54d1aa2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:36.495632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1d2a6ff0-bef9-4728-a762-5485a54d1aa2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:36.502763 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d2a6ff0-bef9-4728-a762-5485a54d1aa2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:36.503228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d2a6ff0-bef9-4728-a762-5485a54d1aa2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:36.503659 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d2a6ff0-bef9-4728-a762-5485a54d1aa2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:36.504355 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d2a6ff0-bef9-4728-a762-5485a54d1aa2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:36.504806 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d2a6ff0-bef9-4728-a762-5485a54d1aa2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:36.505222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1d2a6ff0-bef9-4728-a762-5485a54d1aa2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:36.510408 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:36.539285 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1d2a6ff0-bef9-4728-a762-5485a54d1aa2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050095 Sep 08 14:03:36.544033 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 772/1648] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:36 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:36.676357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3e71b2ba-cef0-4d0e-a3ca-18f75fc4aaef tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:36.677065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3e71b2ba-cef0-4d0e-a3ca-18f75fc4aaef tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:36.684050 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e71b2ba-cef0-4d0e-a3ca-18f75fc4aaef tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:36.684554 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e71b2ba-cef0-4d0e-a3ca-18f75fc4aaef tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:36.685271 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e71b2ba-cef0-4d0e-a3ca-18f75fc4aaef tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:36.685979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e71b2ba-cef0-4d0e-a3ca-18f75fc4aaef tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:36.686456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e71b2ba-cef0-4d0e-a3ca-18f75fc4aaef tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:36.687217 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e71b2ba-cef0-4d0e-a3ca-18f75fc4aaef tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:36.723457 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2f1dd669-684f-423c-84ac-53298ab95cfd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:36.732324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2f1dd669-684f-423c-84ac-53298ab95cfd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.381959 Sep 08 14:03:36.732914 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 877/1649] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:36 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:36.757559 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3e71b2ba-cef0-4d0e-a3ca-18f75fc4aaef tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:36.765898 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3e71b2ba-cef0-4d0e-a3ca-18f75fc4aaef tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.094907 Sep 08 14:03:36.766862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 773/1650] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:36 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:36.776729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c1ad5a90-6870-4fe2-8963-49c274249351 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:36.779679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c1ad5a90-6870-4fe2-8963-49c274249351 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:36.786322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1ad5a90-6870-4fe2-8963-49c274249351 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:36.786862 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1ad5a90-6870-4fe2-8963-49c274249351 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:36.787296 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1ad5a90-6870-4fe2-8963-49c274249351 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:36.787997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1ad5a90-6870-4fe2-8963-49c274249351 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:36.788444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1ad5a90-6870-4fe2-8963-49c274249351 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:36.789061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1ad5a90-6870-4fe2-8963-49c274249351 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:36.838646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c1ad5a90-6870-4fe2-8963-49c274249351 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.065745 Sep 08 14:03:36.839340 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 878/1651] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:36 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:03:37.391285 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1a3073f9-5eb5-45a1-8dec-06f8781f9296 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:37.393609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1a3073f9-5eb5-45a1-8dec-06f8781f9296 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:37.400694 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1a3073f9-5eb5-45a1-8dec-06f8781f9296 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:37.401182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1a3073f9-5eb5-45a1-8dec-06f8781f9296 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:37.401615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1a3073f9-5eb5-45a1-8dec-06f8781f9296 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:37.402347 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1a3073f9-5eb5-45a1-8dec-06f8781f9296 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:37.402800 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1a3073f9-5eb5-45a1-8dec-06f8781f9296 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:37.403221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1a3073f9-5eb5-45a1-8dec-06f8781f9296 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:37.441965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1a3073f9-5eb5-45a1-8dec-06f8781f9296 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055403 Sep 08 14:03:37.451865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 774/1652] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:37 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:37.553265 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-158c58c0-1246-462e-b8e3-cfeb505b4b31 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:37.555484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-158c58c0-1246-462e-b8e3-cfeb505b4b31 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:37.562027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-158c58c0-1246-462e-b8e3-cfeb505b4b31 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:37.563004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-158c58c0-1246-462e-b8e3-cfeb505b4b31 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:37.563519 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-158c58c0-1246-462e-b8e3-cfeb505b4b31 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:37.564170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-158c58c0-1246-462e-b8e3-cfeb505b4b31 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:37.564618 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-158c58c0-1246-462e-b8e3-cfeb505b4b31 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:37.565040 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-158c58c0-1246-462e-b8e3-cfeb505b4b31 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:37.596052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-158c58c0-1246-462e-b8e3-cfeb505b4b31 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046740 Sep 08 14:03:37.596879 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 879/1653] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:37 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:37.756669 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-af8b3946-b0d3-4d39-8587-0c600ee0ef7e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:37.757253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-af8b3946-b0d3-4d39-8587-0c600ee0ef7e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:37.764292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-af8b3946-b0d3-4d39-8587-0c600ee0ef7e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:37.764790 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-af8b3946-b0d3-4d39-8587-0c600ee0ef7e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:37.765847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-af8b3946-b0d3-4d39-8587-0c600ee0ef7e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:37.766588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-af8b3946-b0d3-4d39-8587-0c600ee0ef7e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:37.767041 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-af8b3946-b0d3-4d39-8587-0c600ee0ef7e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:37.767462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-af8b3946-b0d3-4d39-8587-0c600ee0ef7e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:37.789834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d179ad08-cadf-4b3d-8451-d91b652c53e2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:37.790606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d179ad08-cadf-4b3d-8451-d91b652c53e2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:37.797220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d179ad08-cadf-4b3d-8451-d91b652c53e2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:37.798440 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d179ad08-cadf-4b3d-8451-d91b652c53e2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:37.798440 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d179ad08-cadf-4b3d-8451-d91b652c53e2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:37.798861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d179ad08-cadf-4b3d-8451-d91b652c53e2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:37.799390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d179ad08-cadf-4b3d-8451-d91b652c53e2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:37.799861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d179ad08-cadf-4b3d-8451-d91b652c53e2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:37.871872 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d179ad08-cadf-4b3d-8451-d91b652c53e2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:37.880742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d179ad08-cadf-4b3d-8451-d91b652c53e2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.094740 Sep 08 14:03:37.881329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 880/1654] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:37 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:37.890953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b918f5d0-4093-4bb2-965a-c120e83ed088 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:37.893311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b918f5d0-4093-4bb2-965a-c120e83ed088 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:37.900127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b918f5d0-4093-4bb2-965a-c120e83ed088 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:37.900713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b918f5d0-4093-4bb2-965a-c120e83ed088 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:37.901153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b918f5d0-4093-4bb2-965a-c120e83ed088 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:37.901853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b918f5d0-4093-4bb2-965a-c120e83ed088 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:37.902343 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b918f5d0-4093-4bb2-965a-c120e83ed088 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:37.902880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b918f5d0-4093-4bb2-965a-c120e83ed088 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:37.923825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-af8b3946-b0d3-4d39-8587-0c600ee0ef7e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:37.931850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-af8b3946-b0d3-4d39-8587-0c600ee0ef7e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.179557 Sep 08 14:03:37.932452 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 775/1655] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:37 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:37.935022 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b918f5d0-4093-4bb2-965a-c120e83ed088 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052654 Sep 08 14:03:37.935581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 881/1656] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:37 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:38.455247 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a33b5c90-b8cd-4cf2-859e-de38dba499ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:38.457436 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a33b5c90-b8cd-4cf2-859e-de38dba499ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:38.464211 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a33b5c90-b8cd-4cf2-859e-de38dba499ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:38.464705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a33b5c90-b8cd-4cf2-859e-de38dba499ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:38.465142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a33b5c90-b8cd-4cf2-859e-de38dba499ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:38.465860 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a33b5c90-b8cd-4cf2-859e-de38dba499ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:38.466335 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a33b5c90-b8cd-4cf2-859e-de38dba499ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:38.466760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a33b5c90-b8cd-4cf2-859e-de38dba499ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:38.500127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a33b5c90-b8cd-4cf2-859e-de38dba499ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048326 Sep 08 14:03:38.500717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 776/1657] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:38 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:38.609635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-47369346-9825-4033-a32f-711ee87e9a8f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:38.611866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-47369346-9825-4033-a32f-711ee87e9a8f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:38.618471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47369346-9825-4033-a32f-711ee87e9a8f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:38.618963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47369346-9825-4033-a32f-711ee87e9a8f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:38.619404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47369346-9825-4033-a32f-711ee87e9a8f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:38.620104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47369346-9825-4033-a32f-711ee87e9a8f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:38.620557 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47369346-9825-4033-a32f-711ee87e9a8f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:38.620983 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-47369346-9825-4033-a32f-711ee87e9a8f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:38.656567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-47369346-9825-4033-a32f-711ee87e9a8f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050925 Sep 08 14:03:38.657123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 882/1658] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:38 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:38.904970 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-aa4b138b-1939-4fb6-8e10-8d169536cae0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:38.905581 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-aa4b138b-1939-4fb6-8e10-8d169536cae0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:38.912354 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa4b138b-1939-4fb6-8e10-8d169536cae0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:38.913534 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa4b138b-1939-4fb6-8e10-8d169536cae0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:38.913534 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa4b138b-1939-4fb6-8e10-8d169536cae0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:38.913967 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa4b138b-1939-4fb6-8e10-8d169536cae0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:38.914455 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa4b138b-1939-4fb6-8e10-8d169536cae0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:38.914879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa4b138b-1939-4fb6-8e10-8d169536cae0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:38.947513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2a341d00-1c16-423d-824c-1492de6f44e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:38.949737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2a341d00-1c16-423d-824c-1492de6f44e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:38.958498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a341d00-1c16-423d-824c-1492de6f44e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:38.959884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a341d00-1c16-423d-824c-1492de6f44e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:38.960647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a341d00-1c16-423d-824c-1492de6f44e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:38.961443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a341d00-1c16-423d-824c-1492de6f44e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:38.961991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a341d00-1c16-423d-824c-1492de6f44e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:38.962513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a341d00-1c16-423d-824c-1492de6f44e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:38.983055 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-aa4b138b-1939-4fb6-8e10-8d169536cae0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:38.993397 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-aa4b138b-1939-4fb6-8e10-8d169536cae0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.092282 Sep 08 14:03:38.994298 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 777/1659] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:38 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:38.994851 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2a341d00-1c16-423d-824c-1492de6f44e5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051316 Sep 08 14:03:38.995623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 883/1660] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:38 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:03:38.999101 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1f196101-779c-4f65-81e9-d4956117d062 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:39.000291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1f196101-779c-4f65-81e9-d4956117d062 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:39.007228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f196101-779c-4f65-81e9-d4956117d062 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:39.007716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f196101-779c-4f65-81e9-d4956117d062 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:39.008157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f196101-779c-4f65-81e9-d4956117d062 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:39.008857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f196101-779c-4f65-81e9-d4956117d062 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:39.009336 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f196101-779c-4f65-81e9-d4956117d062 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:39.009763 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f196101-779c-4f65-81e9-d4956117d062 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:39.147439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1f196101-779c-4f65-81e9-d4956117d062 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:39.155936 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1f196101-779c-4f65-81e9-d4956117d062 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.160747 Sep 08 14:03:39.156424 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 778/1661] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:38 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:39.514124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-96289021-8dd0-4571-921a-9a8089fc2bf6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:39.516230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-96289021-8dd0-4571-921a-9a8089fc2bf6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:39.522268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-96289021-8dd0-4571-921a-9a8089fc2bf6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:39.522755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-96289021-8dd0-4571-921a-9a8089fc2bf6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:39.523194 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-96289021-8dd0-4571-921a-9a8089fc2bf6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:39.523883 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-96289021-8dd0-4571-921a-9a8089fc2bf6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:39.524337 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-96289021-8dd0-4571-921a-9a8089fc2bf6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:39.524767 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-96289021-8dd0-4571-921a-9a8089fc2bf6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:39.556162 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-96289021-8dd0-4571-921a-9a8089fc2bf6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046076 Sep 08 14:03:39.556747 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 884/1662] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:39 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:39.669210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c2e8803a-e8bb-4905-9d12-f82dde6d1ee1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:39.672010 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c2e8803a-e8bb-4905-9d12-f82dde6d1ee1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:39.678093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2e8803a-e8bb-4905-9d12-f82dde6d1ee1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:39.678614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2e8803a-e8bb-4905-9d12-f82dde6d1ee1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:39.679048 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2e8803a-e8bb-4905-9d12-f82dde6d1ee1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:39.679745 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2e8803a-e8bb-4905-9d12-f82dde6d1ee1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:39.680195 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2e8803a-e8bb-4905-9d12-f82dde6d1ee1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:39.680616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2e8803a-e8bb-4905-9d12-f82dde6d1ee1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:39.710654 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c2e8803a-e8bb-4905-9d12-f82dde6d1ee1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045464 Sep 08 14:03:39.711230 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 779/1663] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:39 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:40.008429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bbfcb629-e44d-4bce-a69a-4c3fd06855cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:40.010608 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bbfcb629-e44d-4bce-a69a-4c3fd06855cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:40.017957 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bbfcb629-e44d-4bce-a69a-4c3fd06855cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:40.018480 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bbfcb629-e44d-4bce-a69a-4c3fd06855cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:40.018861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ce3d2b67-338d-4c78-9089-194dce0ea6fd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:40.019290 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bbfcb629-e44d-4bce-a69a-4c3fd06855cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:40.019476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ce3d2b67-338d-4c78-9089-194dce0ea6fd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:40.020220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bbfcb629-e44d-4bce-a69a-4c3fd06855cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:40.020688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bbfcb629-e44d-4bce-a69a-4c3fd06855cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:40.021121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bbfcb629-e44d-4bce-a69a-4c3fd06855cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:40.026516 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce3d2b67-338d-4c78-9089-194dce0ea6fd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:40.027056 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce3d2b67-338d-4c78-9089-194dce0ea6fd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:40.028383 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce3d2b67-338d-4c78-9089-194dce0ea6fd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:40.029099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce3d2b67-338d-4c78-9089-194dce0ea6fd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:40.029553 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce3d2b67-338d-4c78-9089-194dce0ea6fd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:40.029980 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce3d2b67-338d-4c78-9089-194dce0ea6fd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:40.052228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bbfcb629-e44d-4bce-a69a-4c3fd06855cc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047717 Sep 08 14:03:40.052808 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 885/1664] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:40 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:40.097162 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ce3d2b67-338d-4c78-9089-194dce0ea6fd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:40.105156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ce3d2b67-338d-4c78-9089-194dce0ea6fd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.090735 Sep 08 14:03:40.105721 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 780/1665] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:40 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:40.179676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-61f108f4-f732-40fa-a1fc-62397bb97379 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:40.180209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-61f108f4-f732-40fa-a1fc-62397bb97379 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:40.186128 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61f108f4-f732-40fa-a1fc-62397bb97379 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:40.186635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61f108f4-f732-40fa-a1fc-62397bb97379 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:40.187655 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61f108f4-f732-40fa-a1fc-62397bb97379 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:40.188386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61f108f4-f732-40fa-a1fc-62397bb97379 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:40.188843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61f108f4-f732-40fa-a1fc-62397bb97379 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:40.189267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61f108f4-f732-40fa-a1fc-62397bb97379 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:40.321263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-61f108f4-f732-40fa-a1fc-62397bb97379 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:40.330203 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-61f108f4-f732-40fa-a1fc-62397bb97379 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.154545 Sep 08 14:03:40.330784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 886/1666] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:40 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:40.569378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d7a051e1-ddf6-4bd2-a89d-08c9879ca963 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:40.571489 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d7a051e1-ddf6-4bd2-a89d-08c9879ca963 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:40.577647 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d7a051e1-ddf6-4bd2-a89d-08c9879ca963 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:40.578291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d7a051e1-ddf6-4bd2-a89d-08c9879ca963 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:40.578758 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d7a051e1-ddf6-4bd2-a89d-08c9879ca963 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:40.579463 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d7a051e1-ddf6-4bd2-a89d-08c9879ca963 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:40.579917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d7a051e1-ddf6-4bd2-a89d-08c9879ca963 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:40.580373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d7a051e1-ddf6-4bd2-a89d-08c9879ca963 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:40.614005 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d7a051e1-ddf6-4bd2-a89d-08c9879ca963 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048669 Sep 08 14:03:40.614599 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 781/1667] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:40 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:40.724974 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-93755a67-f483-4043-b93a-3d02f22d2294 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:40.727274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-93755a67-f483-4043-b93a-3d02f22d2294 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:40.733327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93755a67-f483-4043-b93a-3d02f22d2294 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:40.734365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93755a67-f483-4043-b93a-3d02f22d2294 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:40.734810 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93755a67-f483-4043-b93a-3d02f22d2294 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:40.735504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93755a67-f483-4043-b93a-3d02f22d2294 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:40.735961 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93755a67-f483-4043-b93a-3d02f22d2294 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:40.736393 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93755a67-f483-4043-b93a-3d02f22d2294 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:40.766413 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-93755a67-f483-4043-b93a-3d02f22d2294 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045313 Sep 08 14:03:40.767071 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 887/1668] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:40 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:41.065176 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6a4f6e32-cce9-4960-acaf-4aa04193b542 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:41.067548 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6a4f6e32-cce9-4960-acaf-4aa04193b542 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:41.074326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a4f6e32-cce9-4960-acaf-4aa04193b542 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:41.074813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a4f6e32-cce9-4960-acaf-4aa04193b542 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:41.075248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a4f6e32-cce9-4960-acaf-4aa04193b542 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:41.075951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a4f6e32-cce9-4960-acaf-4aa04193b542 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:41.076396 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a4f6e32-cce9-4960-acaf-4aa04193b542 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:41.076816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a4f6e32-cce9-4960-acaf-4aa04193b542 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:41.110111 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6a4f6e32-cce9-4960-acaf-4aa04193b542 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048900 Sep 08 14:03:41.110716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 782/1669] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:41 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:41.128595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9e4026ce-1097-4c9f-9ce4-652b6326d3cd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:41.137266 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9e4026ce-1097-4c9f-9ce4-652b6326d3cd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:41.140453 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e4026ce-1097-4c9f-9ce4-652b6326d3cd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:41.141337 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e4026ce-1097-4c9f-9ce4-652b6326d3cd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:41.142271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e4026ce-1097-4c9f-9ce4-652b6326d3cd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:41.143990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e4026ce-1097-4c9f-9ce4-652b6326d3cd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:41.144608 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e4026ce-1097-4c9f-9ce4-652b6326d3cd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:41.145050 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9e4026ce-1097-4c9f-9ce4-652b6326d3cd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:41.215114 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9e4026ce-1097-4c9f-9ce4-652b6326d3cd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:41.224503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9e4026ce-1097-4c9f-9ce4-652b6326d3cd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.100279 Sep 08 14:03:41.225080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 888/1670] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:41 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:41.353909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3646e83f-37a3-4d57-8277-94b70027b638 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:41.355163 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3646e83f-37a3-4d57-8277-94b70027b638 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:41.361088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3646e83f-37a3-4d57-8277-94b70027b638 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:41.361572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3646e83f-37a3-4d57-8277-94b70027b638 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:41.362018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3646e83f-37a3-4d57-8277-94b70027b638 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:41.362732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3646e83f-37a3-4d57-8277-94b70027b638 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:41.363235 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3646e83f-37a3-4d57-8277-94b70027b638 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:41.363606 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3646e83f-37a3-4d57-8277-94b70027b638 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:41.497467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3646e83f-37a3-4d57-8277-94b70027b638 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:41.505528 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3646e83f-37a3-4d57-8277-94b70027b638 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.155989 Sep 08 14:03:41.506122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 783/1671] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:41 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:41.628631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-250e800c-43f3-41f0-80be-f5f4abc7f42c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:41.630830 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-250e800c-43f3-41f0-80be-f5f4abc7f42c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:41.637039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-250e800c-43f3-41f0-80be-f5f4abc7f42c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:41.637532 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-250e800c-43f3-41f0-80be-f5f4abc7f42c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:41.637978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-250e800c-43f3-41f0-80be-f5f4abc7f42c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:41.638810 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-250e800c-43f3-41f0-80be-f5f4abc7f42c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:41.639384 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-250e800c-43f3-41f0-80be-f5f4abc7f42c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:41.641091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-250e800c-43f3-41f0-80be-f5f4abc7f42c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:41.670991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-250e800c-43f3-41f0-80be-f5f4abc7f42c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047951 Sep 08 14:03:41.671554 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 889/1672] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:41 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:41.779215 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2553dbd0-8ec3-4135-87c7-dab406ff56d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:41.781424 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2553dbd0-8ec3-4135-87c7-dab406ff56d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:41.787768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2553dbd0-8ec3-4135-87c7-dab406ff56d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:41.788259 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2553dbd0-8ec3-4135-87c7-dab406ff56d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:41.788714 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2553dbd0-8ec3-4135-87c7-dab406ff56d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:41.789427 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2553dbd0-8ec3-4135-87c7-dab406ff56d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:41.789880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2553dbd0-8ec3-4135-87c7-dab406ff56d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:41.790346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2553dbd0-8ec3-4135-87c7-dab406ff56d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:41.822937 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2553dbd0-8ec3-4135-87c7-dab406ff56d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047854 Sep 08 14:03:41.823512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 784/1673] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:41 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:42.127177 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1619d641-9432-436e-9e29-f35ac99a4a37 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:42.129573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1619d641-9432-436e-9e29-f35ac99a4a37 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:42.131570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:42.136472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1619d641-9432-436e-9e29-f35ac99a4a37 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:42.136965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1619d641-9432-436e-9e29-f35ac99a4a37 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:42.137409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1619d641-9432-436e-9e29-f35ac99a4a37 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:42.138256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1619d641-9432-436e-9e29-f35ac99a4a37 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:42.145439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1619d641-9432-436e-9e29-f35ac99a4a37 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:42.145439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1619d641-9432-436e-9e29-f35ac99a4a37 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:42.172388 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1619d641-9432-436e-9e29-f35ac99a4a37 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052944 Sep 08 14:03:42.173029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 890/1674] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:42 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:42.247966 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e1d8583e-53b7-4ba6-bbda-11c2e94fc81b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:42.248570 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e1d8583e-53b7-4ba6-bbda-11c2e94fc81b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:42.255121 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1d8583e-53b7-4ba6-bbda-11c2e94fc81b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:42.255608 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1d8583e-53b7-4ba6-bbda-11c2e94fc81b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:42.256040 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1d8583e-53b7-4ba6-bbda-11c2e94fc81b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:42.256736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1d8583e-53b7-4ba6-bbda-11c2e94fc81b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:42.257191 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1d8583e-53b7-4ba6-bbda-11c2e94fc81b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:42.257609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1d8583e-53b7-4ba6-bbda-11c2e94fc81b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:42.322893 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e1d8583e-53b7-4ba6-bbda-11c2e94fc81b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:42.331615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e1d8583e-53b7-4ba6-bbda-11c2e94fc81b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.087930 Sep 08 14:03:42.332185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 785/1675] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:42 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:42.530744 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cbebc75b-e1d9-4b19-b5f4-d633d1d4493b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:42.531301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cbebc75b-e1d9-4b19-b5f4-d633d1d4493b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:42.537338 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbebc75b-e1d9-4b19-b5f4-d633d1d4493b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:42.537822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbebc75b-e1d9-4b19-b5f4-d633d1d4493b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:42.538295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbebc75b-e1d9-4b19-b5f4-d633d1d4493b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:42.538990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbebc75b-e1d9-4b19-b5f4-d633d1d4493b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:42.539437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbebc75b-e1d9-4b19-b5f4-d633d1d4493b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:42.539859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbebc75b-e1d9-4b19-b5f4-d633d1d4493b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:42.674007 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cbebc75b-e1d9-4b19-b5f4-d633d1d4493b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:42.681942 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cbebc75b-e1d9-4b19-b5f4-d633d1d4493b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.155834 Sep 08 14:03:42.689319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 891/1676] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:42 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:42.689576 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-74707993-b1b1-4edd-9088-83240b376e9c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:42.689576 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-74707993-b1b1-4edd-9088-83240b376e9c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:42.692867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74707993-b1b1-4edd-9088-83240b376e9c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:42.693507 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74707993-b1b1-4edd-9088-83240b376e9c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:42.693950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74707993-b1b1-4edd-9088-83240b376e9c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:42.694844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74707993-b1b1-4edd-9088-83240b376e9c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:42.695302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74707993-b1b1-4edd-9088-83240b376e9c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:42.695724 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74707993-b1b1-4edd-9088-83240b376e9c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:42.728032 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-74707993-b1b1-4edd-9088-83240b376e9c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048103 Sep 08 14:03:42.728609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 786/1677] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:42 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:42.836673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d0292f3d-9f79-4ac5-92dc-2e7c64899acc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:42.839475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d0292f3d-9f79-4ac5-92dc-2e7c64899acc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:42.845785 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d0292f3d-9f79-4ac5-92dc-2e7c64899acc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:42.846319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d0292f3d-9f79-4ac5-92dc-2e7c64899acc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:42.846764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d0292f3d-9f79-4ac5-92dc-2e7c64899acc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:42.847463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d0292f3d-9f79-4ac5-92dc-2e7c64899acc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:42.847962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d0292f3d-9f79-4ac5-92dc-2e7c64899acc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:42.848405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d0292f3d-9f79-4ac5-92dc-2e7c64899acc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:42.875644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:42.880555 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d0292f3d-9f79-4ac5-92dc-2e7c64899acc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047902 Sep 08 14:03:42.881140 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 892/1678] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:42 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:43.186042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d83e1462-ed92-4fc2-925d-74a362359f48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:43.188137 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d83e1462-ed92-4fc2-925d-74a362359f48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:43.195291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d83e1462-ed92-4fc2-925d-74a362359f48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:43.195781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d83e1462-ed92-4fc2-925d-74a362359f48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:43.196217 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d83e1462-ed92-4fc2-925d-74a362359f48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:43.196922 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d83e1462-ed92-4fc2-925d-74a362359f48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:43.197374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d83e1462-ed92-4fc2-925d-74a362359f48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:43.197792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d83e1462-ed92-4fc2-925d-74a362359f48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:43.231223 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d83e1462-ed92-4fc2-925d-74a362359f48 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049771 Sep 08 14:03:43.232311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 787/1679] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:43 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:03:43.354927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-390b259e-72cf-4b37-9864-815dc479ea7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:43.355730 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-390b259e-72cf-4b37-9864-815dc479ea7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:43.366391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-390b259e-72cf-4b37-9864-815dc479ea7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:43.366994 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-390b259e-72cf-4b37-9864-815dc479ea7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:43.367510 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-390b259e-72cf-4b37-9864-815dc479ea7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:43.368231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-390b259e-72cf-4b37-9864-815dc479ea7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:43.368697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-390b259e-72cf-4b37-9864-815dc479ea7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:43.369993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-390b259e-72cf-4b37-9864-815dc479ea7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:43.435014 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-390b259e-72cf-4b37-9864-815dc479ea7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:43.443202 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-390b259e-72cf-4b37-9864-815dc479ea7f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.092658 Sep 08 14:03:43.443783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 893/1680] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:43 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:43.706727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-59a6812d-4ecc-4073-af20-db3d850f13dc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:43.707367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-59a6812d-4ecc-4073-af20-db3d850f13dc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:43.713843 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59a6812d-4ecc-4073-af20-db3d850f13dc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:43.714363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59a6812d-4ecc-4073-af20-db3d850f13dc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:43.714799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59a6812d-4ecc-4073-af20-db3d850f13dc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:43.715490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59a6812d-4ecc-4073-af20-db3d850f13dc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:43.715940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59a6812d-4ecc-4073-af20-db3d850f13dc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:43.716362 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59a6812d-4ecc-4073-af20-db3d850f13dc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:43.740654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9fc7dd21-f881-4b19-ab1e-96b469c975af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:43.742954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9fc7dd21-f881-4b19-ab1e-96b469c975af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:43.751161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9fc7dd21-f881-4b19-ab1e-96b469c975af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:43.751655 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9fc7dd21-f881-4b19-ab1e-96b469c975af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:43.752094 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9fc7dd21-f881-4b19-ab1e-96b469c975af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:43.752801 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9fc7dd21-f881-4b19-ab1e-96b469c975af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:43.753254 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9fc7dd21-f881-4b19-ab1e-96b469c975af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:43.753705 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9fc7dd21-f881-4b19-ab1e-96b469c975af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:43.786553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9fc7dd21-f881-4b19-ab1e-96b469c975af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049882 Sep 08 14:03:43.787123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 894/1681] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:43 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:43.851279 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-59a6812d-4ecc-4073-af20-db3d850f13dc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:43.854642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:43.859246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-59a6812d-4ecc-4073-af20-db3d850f13dc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.157072 Sep 08 14:03:43.859819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 788/1682] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:43 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:43.901801 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-24208f3e-6282-4758-834f-3c5385a91e33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:43.903891 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-24208f3e-6282-4758-834f-3c5385a91e33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:43.909855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-24208f3e-6282-4758-834f-3c5385a91e33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:43.910446 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-24208f3e-6282-4758-834f-3c5385a91e33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:43.910888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-24208f3e-6282-4758-834f-3c5385a91e33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:43.911574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-24208f3e-6282-4758-834f-3c5385a91e33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:43.912027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-24208f3e-6282-4758-834f-3c5385a91e33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:43.912458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-24208f3e-6282-4758-834f-3c5385a91e33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:43.942556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-24208f3e-6282-4758-834f-3c5385a91e33 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044645 Sep 08 14:03:43.943127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 895/1683] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:43 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:44.244973 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-190963fe-aa72-486e-88b5-9066749c2664 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:44.247344 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-190963fe-aa72-486e-88b5-9066749c2664 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:44.254621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-190963fe-aa72-486e-88b5-9066749c2664 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:44.255107 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-190963fe-aa72-486e-88b5-9066749c2664 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:44.255555 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-190963fe-aa72-486e-88b5-9066749c2664 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:44.256241 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-190963fe-aa72-486e-88b5-9066749c2664 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:44.256692 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-190963fe-aa72-486e-88b5-9066749c2664 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:44.257117 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-190963fe-aa72-486e-88b5-9066749c2664 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:44.286175 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-190963fe-aa72-486e-88b5-9066749c2664 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045322 Sep 08 14:03:44.286735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 789/1684] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:44 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:44.467186 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-029a43b2-1a52-439c-8dcc-1d4adf3700c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:44.467822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-029a43b2-1a52-439c-8dcc-1d4adf3700c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:44.474407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-029a43b2-1a52-439c-8dcc-1d4adf3700c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:44.474895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-029a43b2-1a52-439c-8dcc-1d4adf3700c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:44.475332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-029a43b2-1a52-439c-8dcc-1d4adf3700c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:44.476127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-029a43b2-1a52-439c-8dcc-1d4adf3700c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:44.476622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-029a43b2-1a52-439c-8dcc-1d4adf3700c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:44.477029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-029a43b2-1a52-439c-8dcc-1d4adf3700c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:44.551002 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-029a43b2-1a52-439c-8dcc-1d4adf3700c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:44.559147 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-029a43b2-1a52-439c-8dcc-1d4adf3700c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.096254 Sep 08 14:03:44.559734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 896/1685] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:44 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:44.799200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c617e9c3-afa5-47c8-acf9-d7ed4c622353 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:44.801477 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c617e9c3-afa5-47c8-acf9-d7ed4c622353 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:44.808262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c617e9c3-afa5-47c8-acf9-d7ed4c622353 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:44.808760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c617e9c3-afa5-47c8-acf9-d7ed4c622353 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:44.809194 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c617e9c3-afa5-47c8-acf9-d7ed4c622353 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:44.809928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c617e9c3-afa5-47c8-acf9-d7ed4c622353 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:44.810419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c617e9c3-afa5-47c8-acf9-d7ed4c622353 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:44.812412 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c617e9c3-afa5-47c8-acf9-d7ed4c622353 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:44.847532 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c617e9c3-afa5-47c8-acf9-d7ed4c622353 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053422 Sep 08 14:03:44.848123 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 790/1686] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:44 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:44.886263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-68b878b4-f7fa-40fb-980e-09166413865e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:44.886263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-68b878b4-f7fa-40fb-980e-09166413865e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:44.889931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68b878b4-f7fa-40fb-980e-09166413865e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:44.890447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68b878b4-f7fa-40fb-980e-09166413865e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:44.891436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68b878b4-f7fa-40fb-980e-09166413865e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:44.892136 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68b878b4-f7fa-40fb-980e-09166413865e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:44.892589 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68b878b4-f7fa-40fb-980e-09166413865e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:44.893012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68b878b4-f7fa-40fb-980e-09166413865e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:44.956142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cd7eb6d1-aa7a-4444-a1b8-655eadf2f48c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:44.958755 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cd7eb6d1-aa7a-4444-a1b8-655eadf2f48c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:44.964984 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd7eb6d1-aa7a-4444-a1b8-655eadf2f48c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:44.965701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd7eb6d1-aa7a-4444-a1b8-655eadf2f48c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:44.966265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd7eb6d1-aa7a-4444-a1b8-655eadf2f48c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:44.967248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd7eb6d1-aa7a-4444-a1b8-655eadf2f48c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:44.967799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd7eb6d1-aa7a-4444-a1b8-655eadf2f48c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:44.968293 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd7eb6d1-aa7a-4444-a1b8-655eadf2f48c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:45.001834 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cd7eb6d1-aa7a-4444-a1b8-655eadf2f48c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050401 Sep 08 14:03:45.002447 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 791/1687] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:44 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:45.035945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-68b878b4-f7fa-40fb-980e-09166413865e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:45.043845 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-68b878b4-f7fa-40fb-980e-09166413865e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.164714 Sep 08 14:03:45.044341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 897/1688] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:44 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:45.298112 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-90e4f6c7-a552-4004-af98-bcc9e7812358 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:45.300200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-90e4f6c7-a552-4004-af98-bcc9e7812358 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:45.306320 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-90e4f6c7-a552-4004-af98-bcc9e7812358 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:45.306802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-90e4f6c7-a552-4004-af98-bcc9e7812358 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:45.307233 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-90e4f6c7-a552-4004-af98-bcc9e7812358 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:45.307926 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-90e4f6c7-a552-4004-af98-bcc9e7812358 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:45.308373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-90e4f6c7-a552-4004-af98-bcc9e7812358 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:45.308792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-90e4f6c7-a552-4004-af98-bcc9e7812358 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:45.339580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-90e4f6c7-a552-4004-af98-bcc9e7812358 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045584 Sep 08 14:03:45.340142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 792/1689] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:45 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:45.593751 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-50a115d8-394d-4c34-9634-3867f9050e46 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:45.593751 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-50a115d8-394d-4c34-9634-3867f9050e46 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:45.593751 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50a115d8-394d-4c34-9634-3867f9050e46 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:45.593751 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50a115d8-394d-4c34-9634-3867f9050e46 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:45.594717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50a115d8-394d-4c34-9634-3867f9050e46 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:45.594717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50a115d8-394d-4c34-9634-3867f9050e46 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:45.594717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50a115d8-394d-4c34-9634-3867f9050e46 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:45.594717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50a115d8-394d-4c34-9634-3867f9050e46 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:45.658129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-50a115d8-394d-4c34-9634-3867f9050e46 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:45.665987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-50a115d8-394d-4c34-9634-3867f9050e46 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.085016 Sep 08 14:03:45.666612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 898/1690] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:45 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:45.860780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-93dde547-7810-4ca3-b0e6-08d97dbf11f0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:45.862885 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-93dde547-7810-4ca3-b0e6-08d97dbf11f0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:45.869506 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-93dde547-7810-4ca3-b0e6-08d97dbf11f0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:45.869994 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-93dde547-7810-4ca3-b0e6-08d97dbf11f0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:45.870462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-93dde547-7810-4ca3-b0e6-08d97dbf11f0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:45.871432 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-93dde547-7810-4ca3-b0e6-08d97dbf11f0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:45.871881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-93dde547-7810-4ca3-b0e6-08d97dbf11f0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:45.872301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-93dde547-7810-4ca3-b0e6-08d97dbf11f0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:45.902318 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-93dde547-7810-4ca3-b0e6-08d97dbf11f0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046124 Sep 08 14:03:45.902878 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 793/1691] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:45 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:46.015152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-85542740-6839-4cf7-8934-23ea224351c3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:46.017538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-85542740-6839-4cf7-8934-23ea224351c3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:46.024204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85542740-6839-4cf7-8934-23ea224351c3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:46.024695 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85542740-6839-4cf7-8934-23ea224351c3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:46.025133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85542740-6839-4cf7-8934-23ea224351c3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:46.025830 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85542740-6839-4cf7-8934-23ea224351c3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:46.026319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85542740-6839-4cf7-8934-23ea224351c3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:46.026747 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85542740-6839-4cf7-8934-23ea224351c3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:46.058334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-85542740-6839-4cf7-8934-23ea224351c3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047535 Sep 08 14:03:46.058965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 899/1692] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:46 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:46.066833 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1b52fe90-32d0-4121-b645-3fe93be5cc70 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:46.067871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1b52fe90-32d0-4121-b645-3fe93be5cc70 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:46.074110 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b52fe90-32d0-4121-b645-3fe93be5cc70 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:46.074633 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b52fe90-32d0-4121-b645-3fe93be5cc70 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:46.075070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b52fe90-32d0-4121-b645-3fe93be5cc70 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:46.075775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b52fe90-32d0-4121-b645-3fe93be5cc70 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:46.076237 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b52fe90-32d0-4121-b645-3fe93be5cc70 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:46.076692 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1b52fe90-32d0-4121-b645-3fe93be5cc70 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:46.220443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1b52fe90-32d0-4121-b645-3fe93be5cc70 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:46.234228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1b52fe90-32d0-4121-b645-3fe93be5cc70 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.171279 Sep 08 14:03:46.235456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 794/1693] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:46 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:46.353192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-408760ea-0406-4d53-bd68-afbd15fb60a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:46.355373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-408760ea-0406-4d53-bd68-afbd15fb60a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:46.362271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-408760ea-0406-4d53-bd68-afbd15fb60a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:46.362805 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-408760ea-0406-4d53-bd68-afbd15fb60a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:46.363244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-408760ea-0406-4d53-bd68-afbd15fb60a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:46.363992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-408760ea-0406-4d53-bd68-afbd15fb60a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:46.364506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-408760ea-0406-4d53-bd68-afbd15fb60a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:46.364997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-408760ea-0406-4d53-bd68-afbd15fb60a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:46.396044 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-408760ea-0406-4d53-bd68-afbd15fb60a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046710 Sep 08 14:03:46.396567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 900/1694] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:46 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:46.689946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e4744098-3085-442d-aa8b-74640d8fb162 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:46.690519 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e4744098-3085-442d-aa8b-74640d8fb162 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:46.696905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4744098-3085-442d-aa8b-74640d8fb162 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:46.697388 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4744098-3085-442d-aa8b-74640d8fb162 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:46.697884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4744098-3085-442d-aa8b-74640d8fb162 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:46.698615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4744098-3085-442d-aa8b-74640d8fb162 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:46.699068 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4744098-3085-442d-aa8b-74640d8fb162 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:46.699481 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4744098-3085-442d-aa8b-74640d8fb162 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:46.768385 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e4744098-3085-442d-aa8b-74640d8fb162 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:46.776412 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e4744098-3085-442d-aa8b-74640d8fb162 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.090490 Sep 08 14:03:46.776980 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 795/1695] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:46 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:46.915385 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3c7bc518-7884-4854-85f3-b2c810dc34fe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:46.917573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3c7bc518-7884-4854-85f3-b2c810dc34fe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:46.924643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3c7bc518-7884-4854-85f3-b2c810dc34fe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:46.925146 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3c7bc518-7884-4854-85f3-b2c810dc34fe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:46.925593 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3c7bc518-7884-4854-85f3-b2c810dc34fe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:46.926361 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3c7bc518-7884-4854-85f3-b2c810dc34fe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:46.926871 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3c7bc518-7884-4854-85f3-b2c810dc34fe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:46.927235 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3c7bc518-7884-4854-85f3-b2c810dc34fe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:46.963079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3c7bc518-7884-4854-85f3-b2c810dc34fe tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051452 Sep 08 14:03:46.963566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 901/1696] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:46 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:47.077287 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-17d2c028-74cb-423b-8289-e99566e3326b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:47.080395 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-17d2c028-74cb-423b-8289-e99566e3326b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:47.087311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-17d2c028-74cb-423b-8289-e99566e3326b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:47.087803 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-17d2c028-74cb-423b-8289-e99566e3326b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:47.088244 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-17d2c028-74cb-423b-8289-e99566e3326b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:47.088948 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-17d2c028-74cb-423b-8289-e99566e3326b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:47.089407 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-17d2c028-74cb-423b-8289-e99566e3326b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:47.089828 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-17d2c028-74cb-423b-8289-e99566e3326b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:47.125563 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-17d2c028-74cb-423b-8289-e99566e3326b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051682 Sep 08 14:03:47.126178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 796/1697] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:47 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:47.263173 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8cfe0b76-877f-4a22-b085-5d629b48f0a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:47.263684 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8cfe0b76-877f-4a22-b085-5d629b48f0a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:47.270829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8cfe0b76-877f-4a22-b085-5d629b48f0a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:47.271378 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8cfe0b76-877f-4a22-b085-5d629b48f0a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:47.271953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8cfe0b76-877f-4a22-b085-5d629b48f0a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:47.272678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8cfe0b76-877f-4a22-b085-5d629b48f0a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:47.273129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8cfe0b76-877f-4a22-b085-5d629b48f0a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:47.273558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8cfe0b76-877f-4a22-b085-5d629b48f0a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:47.411359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c6a7b3a5-ab23-4532-961f-b6618f1b57fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:47.413534 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c6a7b3a5-ab23-4532-961f-b6618f1b57fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:47.420634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6a7b3a5-ab23-4532-961f-b6618f1b57fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:47.421114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6a7b3a5-ab23-4532-961f-b6618f1b57fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:47.421546 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6a7b3a5-ab23-4532-961f-b6618f1b57fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:47.422809 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6a7b3a5-ab23-4532-961f-b6618f1b57fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:47.423255 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6a7b3a5-ab23-4532-961f-b6618f1b57fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:47.423557 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8cfe0b76-877f-4a22-b085-5d629b48f0a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:47.423906 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6a7b3a5-ab23-4532-961f-b6618f1b57fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:47.431973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8cfe0b76-877f-4a22-b085-5d629b48f0a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.173283 Sep 08 14:03:47.436896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 902/1698] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:47 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:47.457956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c6a7b3a5-ab23-4532-961f-b6618f1b57fc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053202 Sep 08 14:03:47.458742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 797/1699] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:47 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:47.801930 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bd5ccc2e-c763-4178-b06a-2645c9425b93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:47.802587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bd5ccc2e-c763-4178-b06a-2645c9425b93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:47.808983 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd5ccc2e-c763-4178-b06a-2645c9425b93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:47.809190 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd5ccc2e-c763-4178-b06a-2645c9425b93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:47.809992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd5ccc2e-c763-4178-b06a-2645c9425b93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:47.810379 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd5ccc2e-c763-4178-b06a-2645c9425b93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:47.811280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd5ccc2e-c763-4178-b06a-2645c9425b93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:47.811280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd5ccc2e-c763-4178-b06a-2645c9425b93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:47.878623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bd5ccc2e-c763-4178-b06a-2645c9425b93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:47.886790 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bd5ccc2e-c763-4178-b06a-2645c9425b93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.090632 Sep 08 14:03:47.887373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 903/1700] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:47 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:48.043496 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7fc4de7d-c5d4-4678-b5c8-dd1ec35cda2f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:48.045751 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7fc4de7d-c5d4-4678-b5c8-dd1ec35cda2f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:48.052028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7fc4de7d-c5d4-4678-b5c8-dd1ec35cda2f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:48.052515 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7fc4de7d-c5d4-4678-b5c8-dd1ec35cda2f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:48.052949 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7fc4de7d-c5d4-4678-b5c8-dd1ec35cda2f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:48.053654 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7fc4de7d-c5d4-4678-b5c8-dd1ec35cda2f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:48.054104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7fc4de7d-c5d4-4678-b5c8-dd1ec35cda2f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:48.054550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7fc4de7d-c5d4-4678-b5c8-dd1ec35cda2f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:48.086083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7fc4de7d-c5d4-4678-b5c8-dd1ec35cda2f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.113798 Sep 08 14:03:48.086826 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 798/1701] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:47 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:48.139444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fed7e9e5-eed6-4f81-ac6f-d8b9027b7e6c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:48.142404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fed7e9e5-eed6-4f81-ac6f-d8b9027b7e6c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:48.148544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fed7e9e5-eed6-4f81-ac6f-d8b9027b7e6c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:48.149030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fed7e9e5-eed6-4f81-ac6f-d8b9027b7e6c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:48.149482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fed7e9e5-eed6-4f81-ac6f-d8b9027b7e6c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:48.150197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fed7e9e5-eed6-4f81-ac6f-d8b9027b7e6c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:48.150654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fed7e9e5-eed6-4f81-ac6f-d8b9027b7e6c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:48.151083 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fed7e9e5-eed6-4f81-ac6f-d8b9027b7e6c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:48.184330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fed7e9e5-eed6-4f81-ac6f-d8b9027b7e6c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049263 Sep 08 14:03:48.185105 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 904/1702] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:48 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:48.457258 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-db6cd37a-9333-4939-8c60-b42989117102 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:48.457788 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-db6cd37a-9333-4939-8c60-b42989117102 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:48.463874 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-db6cd37a-9333-4939-8c60-b42989117102 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:48.464360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-db6cd37a-9333-4939-8c60-b42989117102 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:48.464834 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-db6cd37a-9333-4939-8c60-b42989117102 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:48.465531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-db6cd37a-9333-4939-8c60-b42989117102 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:48.465985 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-db6cd37a-9333-4939-8c60-b42989117102 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:48.466443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-db6cd37a-9333-4939-8c60-b42989117102 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:48.472983 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bc29b800-02ff-4527-94ed-93c17146ca1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:48.475231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bc29b800-02ff-4527-94ed-93c17146ca1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:48.481694 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bc29b800-02ff-4527-94ed-93c17146ca1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:48.482223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bc29b800-02ff-4527-94ed-93c17146ca1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:48.482692 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bc29b800-02ff-4527-94ed-93c17146ca1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:48.483401 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bc29b800-02ff-4527-94ed-93c17146ca1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:48.483855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bc29b800-02ff-4527-94ed-93c17146ca1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:48.484277 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bc29b800-02ff-4527-94ed-93c17146ca1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:48.520553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bc29b800-02ff-4527-94ed-93c17146ca1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051431 Sep 08 14:03:48.521143 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 905/1703] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:48 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:03:48.618555 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-db6cd37a-9333-4939-8c60-b42989117102 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:48.627495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-db6cd37a-9333-4939-8c60-b42989117102 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.174761 Sep 08 14:03:48.628064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 799/1704] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:48 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:48.912926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d80569e5-1112-4e29-9072-918dad724a84 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:48.913523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d80569e5-1112-4e29-9072-918dad724a84 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:48.920176 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d80569e5-1112-4e29-9072-918dad724a84 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:48.920752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d80569e5-1112-4e29-9072-918dad724a84 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:48.921188 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d80569e5-1112-4e29-9072-918dad724a84 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:48.922436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d80569e5-1112-4e29-9072-918dad724a84 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:48.922886 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d80569e5-1112-4e29-9072-918dad724a84 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:48.923307 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d80569e5-1112-4e29-9072-918dad724a84 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:48.996700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d80569e5-1112-4e29-9072-918dad724a84 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:49.006280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d80569e5-1112-4e29-9072-918dad724a84 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.097411 Sep 08 14:03:49.006867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 906/1705] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:48 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:49.100106 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-29baccc5-d8f8-4dda-a2c8-3690dff69915 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:49.102343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-29baccc5-d8f8-4dda-a2c8-3690dff69915 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:49.108906 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29baccc5-d8f8-4dda-a2c8-3690dff69915 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:49.109964 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29baccc5-d8f8-4dda-a2c8-3690dff69915 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:49.110432 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29baccc5-d8f8-4dda-a2c8-3690dff69915 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:49.111212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29baccc5-d8f8-4dda-a2c8-3690dff69915 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:49.111734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29baccc5-d8f8-4dda-a2c8-3690dff69915 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:49.112190 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29baccc5-d8f8-4dda-a2c8-3690dff69915 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:49.145200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-29baccc5-d8f8-4dda-a2c8-3690dff69915 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049077 Sep 08 14:03:49.145812 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 800/1706] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:49 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:49.197283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-10098bd9-fa2b-4417-9d2c-fee7e459c855 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:49.199389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-10098bd9-fa2b-4417-9d2c-fee7e459c855 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:49.206020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-10098bd9-fa2b-4417-9d2c-fee7e459c855 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:49.206536 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-10098bd9-fa2b-4417-9d2c-fee7e459c855 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:49.206969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-10098bd9-fa2b-4417-9d2c-fee7e459c855 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:49.207661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-10098bd9-fa2b-4417-9d2c-fee7e459c855 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:49.208109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-10098bd9-fa2b-4417-9d2c-fee7e459c855 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:49.208525 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-10098bd9-fa2b-4417-9d2c-fee7e459c855 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:49.240027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-10098bd9-fa2b-4417-9d2c-fee7e459c855 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046873 Sep 08 14:03:49.240595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 907/1707] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:49 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:49.534628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-231b02bb-c736-44c1-9490-ec4915d2dc89 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:49.536718 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-231b02bb-c736-44c1-9490-ec4915d2dc89 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:49.542753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-231b02bb-c736-44c1-9490-ec4915d2dc89 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:49.543234 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-231b02bb-c736-44c1-9490-ec4915d2dc89 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:49.543665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-231b02bb-c736-44c1-9490-ec4915d2dc89 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:49.544348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-231b02bb-c736-44c1-9490-ec4915d2dc89 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:49.544796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-231b02bb-c736-44c1-9490-ec4915d2dc89 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:49.545218 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-231b02bb-c736-44c1-9490-ec4915d2dc89 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:49.574513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-231b02bb-c736-44c1-9490-ec4915d2dc89 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045021 Sep 08 14:03:49.575150 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 801/1708] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:49 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:49.652389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-04a6e8ae-0179-461d-919e-35feccda81a0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:49.653009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-04a6e8ae-0179-461d-919e-35feccda81a0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:49.659917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04a6e8ae-0179-461d-919e-35feccda81a0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:49.660418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04a6e8ae-0179-461d-919e-35feccda81a0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:49.660859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04a6e8ae-0179-461d-919e-35feccda81a0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:49.661556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04a6e8ae-0179-461d-919e-35feccda81a0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:49.662042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04a6e8ae-0179-461d-919e-35feccda81a0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:49.662503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-04a6e8ae-0179-461d-919e-35feccda81a0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:49.820813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-04a6e8ae-0179-461d-919e-35feccda81a0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:49.831262 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-04a6e8ae-0179-461d-919e-35feccda81a0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.182370 Sep 08 14:03:49.832408 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 908/1709] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:49 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:50.035550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ce3ee57c-50e4-43a2-a7d6-b3111e61acd3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:50.036145 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ce3ee57c-50e4-43a2-a7d6-b3111e61acd3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:50.042161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce3ee57c-50e4-43a2-a7d6-b3111e61acd3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:50.042638 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce3ee57c-50e4-43a2-a7d6-b3111e61acd3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:50.043070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce3ee57c-50e4-43a2-a7d6-b3111e61acd3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:50.043747 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce3ee57c-50e4-43a2-a7d6-b3111e61acd3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:50.044192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce3ee57c-50e4-43a2-a7d6-b3111e61acd3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:50.044608 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce3ee57c-50e4-43a2-a7d6-b3111e61acd3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:50.113785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ce3ee57c-50e4-43a2-a7d6-b3111e61acd3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:50.125442 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ce3ee57c-50e4-43a2-a7d6-b3111e61acd3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.093178 Sep 08 14:03:50.126033 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 802/1710] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:50 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:50.157476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a719c673-6d20-488c-918a-ef2eab132020 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:50.159639 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a719c673-6d20-488c-918a-ef2eab132020 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:50.165579 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a719c673-6d20-488c-918a-ef2eab132020 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:50.166063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a719c673-6d20-488c-918a-ef2eab132020 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:50.166529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a719c673-6d20-488c-918a-ef2eab132020 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:50.167212 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a719c673-6d20-488c-918a-ef2eab132020 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:50.167656 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a719c673-6d20-488c-918a-ef2eab132020 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:50.168074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a719c673-6d20-488c-918a-ef2eab132020 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:50.182033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:50.199854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a719c673-6d20-488c-918a-ef2eab132020 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046318 Sep 08 14:03:50.200595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 909/1711] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:50 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:50.253212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5f0b6729-ca55-4969-ab09-1f28c4d4a5a8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:50.255475 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5f0b6729-ca55-4969-ab09-1f28c4d4a5a8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:50.261666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5f0b6729-ca55-4969-ab09-1f28c4d4a5a8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:50.262164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5f0b6729-ca55-4969-ab09-1f28c4d4a5a8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:50.262602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5f0b6729-ca55-4969-ab09-1f28c4d4a5a8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:50.263888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5f0b6729-ca55-4969-ab09-1f28c4d4a5a8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:50.263888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5f0b6729-ca55-4969-ab09-1f28c4d4a5a8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:50.264127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5f0b6729-ca55-4969-ab09-1f28c4d4a5a8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:50.293940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5f0b6729-ca55-4969-ab09-1f28c4d4a5a8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045090 Sep 08 14:03:50.294609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 803/1712] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:50 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:50.588373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d29c4a8d-4ca7-46a3-be72-d78ae6ae1427 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:50.590573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d29c4a8d-4ca7-46a3-be72-d78ae6ae1427 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:50.598827 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d29c4a8d-4ca7-46a3-be72-d78ae6ae1427 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:50.599157 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d29c4a8d-4ca7-46a3-be72-d78ae6ae1427 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:50.599607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d29c4a8d-4ca7-46a3-be72-d78ae6ae1427 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:50.600319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d29c4a8d-4ca7-46a3-be72-d78ae6ae1427 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:50.600781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d29c4a8d-4ca7-46a3-be72-d78ae6ae1427 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:50.601216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d29c4a8d-4ca7-46a3-be72-d78ae6ae1427 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:50.635327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d29c4a8d-4ca7-46a3-be72-d78ae6ae1427 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051304 Sep 08 14:03:50.635940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 910/1713] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:50 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:50.855569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5e98e349-2f58-42cf-805b-160d6cef1179 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:50.856103 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5e98e349-2f58-42cf-805b-160d6cef1179 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:50.862951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e98e349-2f58-42cf-805b-160d6cef1179 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:50.863439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e98e349-2f58-42cf-805b-160d6cef1179 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:50.863930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e98e349-2f58-42cf-805b-160d6cef1179 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:50.864639 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e98e349-2f58-42cf-805b-160d6cef1179 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:50.865094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e98e349-2f58-42cf-805b-160d6cef1179 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:50.865522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e98e349-2f58-42cf-805b-160d6cef1179 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:51.010523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5e98e349-2f58-42cf-805b-160d6cef1179 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:51.019403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5e98e349-2f58-42cf-805b-160d6cef1179 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.167824 Sep 08 14:03:51.020152 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 804/1714] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:50 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:51.149199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-019ae86e-f598-428f-b843-8d12576b7203 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:51.149717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-019ae86e-f598-428f-b843-8d12576b7203 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:51.156449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-019ae86e-f598-428f-b843-8d12576b7203 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:51.156939 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-019ae86e-f598-428f-b843-8d12576b7203 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:51.157455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-019ae86e-f598-428f-b843-8d12576b7203 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:51.158182 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-019ae86e-f598-428f-b843-8d12576b7203 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:51.158645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-019ae86e-f598-428f-b843-8d12576b7203 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:51.159071 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-019ae86e-f598-428f-b843-8d12576b7203 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:51.214725 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-84d49111-59c4-4500-b603-6ab2fa49afb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:51.218049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-84d49111-59c4-4500-b603-6ab2fa49afb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:51.227048 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84d49111-59c4-4500-b603-6ab2fa49afb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:51.227539 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84d49111-59c4-4500-b603-6ab2fa49afb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:51.227965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84d49111-59c4-4500-b603-6ab2fa49afb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:51.228432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-019ae86e-f598-428f-b843-8d12576b7203 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:51.228865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84d49111-59c4-4500-b603-6ab2fa49afb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:51.229101 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84d49111-59c4-4500-b603-6ab2fa49afb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:51.229525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84d49111-59c4-4500-b603-6ab2fa49afb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:51.241219 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-019ae86e-f598-428f-b843-8d12576b7203 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.095707 Sep 08 14:03:51.242129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 911/1715] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:51 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:51.250275 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:51.265167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-84d49111-59c4-4500-b603-6ab2fa49afb5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056182 Sep 08 14:03:51.266470 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 805/1716] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:51 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:51.307482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-abd2bf44-e778-47a1-bf66-0d1dd3515e2d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:51.309634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-abd2bf44-e778-47a1-bf66-0d1dd3515e2d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:51.316319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-abd2bf44-e778-47a1-bf66-0d1dd3515e2d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:51.316807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-abd2bf44-e778-47a1-bf66-0d1dd3515e2d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:51.317244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-abd2bf44-e778-47a1-bf66-0d1dd3515e2d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:51.317968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-abd2bf44-e778-47a1-bf66-0d1dd3515e2d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:51.318458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-abd2bf44-e778-47a1-bf66-0d1dd3515e2d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:51.318883 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-abd2bf44-e778-47a1-bf66-0d1dd3515e2d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:51.354681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-abd2bf44-e778-47a1-bf66-0d1dd3515e2d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050673 Sep 08 14:03:51.355254 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 912/1717] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:51 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:51.650706 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-be0c0d9e-89a6-4b34-add4-f4fcf796985b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:51.652891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-be0c0d9e-89a6-4b34-add4-f4fcf796985b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:51.659614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be0c0d9e-89a6-4b34-add4-f4fcf796985b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:51.660092 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be0c0d9e-89a6-4b34-add4-f4fcf796985b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:51.660534 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be0c0d9e-89a6-4b34-add4-f4fcf796985b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:51.661235 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be0c0d9e-89a6-4b34-add4-f4fcf796985b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:51.661685 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be0c0d9e-89a6-4b34-add4-f4fcf796985b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:51.662110 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be0c0d9e-89a6-4b34-add4-f4fcf796985b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:51.705390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-be0c0d9e-89a6-4b34-add4-f4fcf796985b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.059747 Sep 08 14:03:51.705996 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 806/1718] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:51 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:52.043866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f8cf3d25-5166-4a1c-ad47-dec291f4fc48 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:52.044500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f8cf3d25-5166-4a1c-ad47-dec291f4fc48 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:52.051306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8cf3d25-5166-4a1c-ad47-dec291f4fc48 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:52.051794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8cf3d25-5166-4a1c-ad47-dec291f4fc48 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:52.052230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8cf3d25-5166-4a1c-ad47-dec291f4fc48 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:52.052935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8cf3d25-5166-4a1c-ad47-dec291f4fc48 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:52.053389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8cf3d25-5166-4a1c-ad47-dec291f4fc48 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:52.053813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8cf3d25-5166-4a1c-ad47-dec291f4fc48 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:52.219469 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f8cf3d25-5166-4a1c-ad47-dec291f4fc48 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:52.227926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f8cf3d25-5166-4a1c-ad47-dec291f4fc48 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.187928 Sep 08 14:03:52.228670 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 913/1719] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:52 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:52.268742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-67e65d0d-c03d-4827-824b-3ab93ad3ef54 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:52.269455 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-67e65d0d-c03d-4827-824b-3ab93ad3ef54 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:52.277856 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67e65d0d-c03d-4827-824b-3ab93ad3ef54 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:52.278819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67e65d0d-c03d-4827-824b-3ab93ad3ef54 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:52.279531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67e65d0d-c03d-4827-824b-3ab93ad3ef54 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:52.280652 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67e65d0d-c03d-4827-824b-3ab93ad3ef54 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:52.281415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67e65d0d-c03d-4827-824b-3ab93ad3ef54 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:52.281619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5d6f4099-d1a2-4ef3-a394-aa613fcdca73 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:52.282272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67e65d0d-c03d-4827-824b-3ab93ad3ef54 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:52.285421 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5d6f4099-d1a2-4ef3-a394-aa613fcdca73 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:52.295793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5d6f4099-d1a2-4ef3-a394-aa613fcdca73 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:52.296624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5d6f4099-d1a2-4ef3-a394-aa613fcdca73 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:52.297335 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5d6f4099-d1a2-4ef3-a394-aa613fcdca73 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:52.298521 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5d6f4099-d1a2-4ef3-a394-aa613fcdca73 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:52.299856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5d6f4099-d1a2-4ef3-a394-aa613fcdca73 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:52.300027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5d6f4099-d1a2-4ef3-a394-aa613fcdca73 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:52.354788 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5d6f4099-d1a2-4ef3-a394-aa613fcdca73 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.079631 Sep 08 14:03:52.355744 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 914/1720] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:52 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:52.366759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-67e65d0d-c03d-4827-824b-3ab93ad3ef54 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:52.371998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8d2b8393-7991-4c29-93fc-34f004806921 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:52.375813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-67e65d0d-c03d-4827-824b-3ab93ad3ef54 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.112294 Sep 08 14:03:52.376399 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 807/1721] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:52 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:52.376797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8d2b8393-7991-4c29-93fc-34f004806921 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:52.387219 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8d2b8393-7991-4c29-93fc-34f004806921 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:52.388000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8d2b8393-7991-4c29-93fc-34f004806921 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:52.388720 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8d2b8393-7991-4c29-93fc-34f004806921 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:52.389943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8d2b8393-7991-4c29-93fc-34f004806921 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:52.391515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8d2b8393-7991-4c29-93fc-34f004806921 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:52.392136 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8d2b8393-7991-4c29-93fc-34f004806921 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:52.456965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8d2b8393-7991-4c29-93fc-34f004806921 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.092177 Sep 08 14:03:52.457993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 915/1722] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:52 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:52.717614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8635cb66-94a6-4e2d-99d9-499027a3d33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:52.720457 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8635cb66-94a6-4e2d-99d9-499027a3d33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:52.726664 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8635cb66-94a6-4e2d-99d9-499027a3d33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:52.727182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8635cb66-94a6-4e2d-99d9-499027a3d33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:52.727611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8635cb66-94a6-4e2d-99d9-499027a3d33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:52.728340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8635cb66-94a6-4e2d-99d9-499027a3d33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:52.728827 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8635cb66-94a6-4e2d-99d9-499027a3d33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:52.729251 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8635cb66-94a6-4e2d-99d9-499027a3d33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:52.762378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8635cb66-94a6-4e2d-99d9-499027a3d33d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048742 Sep 08 14:03:52.763057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 808/1723] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:52 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:03:53.254168 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0c3ac280-d0b2-4063-b203-e84f76b1d342 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:53.254827 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0c3ac280-d0b2-4063-b203-e84f76b1d342 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:53.261602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0c3ac280-d0b2-4063-b203-e84f76b1d342 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:53.262373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0c3ac280-d0b2-4063-b203-e84f76b1d342 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:53.262819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0c3ac280-d0b2-4063-b203-e84f76b1d342 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:53.263528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0c3ac280-d0b2-4063-b203-e84f76b1d342 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:53.263987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0c3ac280-d0b2-4063-b203-e84f76b1d342 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:53.264418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0c3ac280-d0b2-4063-b203-e84f76b1d342 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:53.369249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f02bc115-4cf0-46e6-bade-9366f14c4f46 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:53.372522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f02bc115-4cf0-46e6-bade-9366f14c4f46 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:53.380785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f02bc115-4cf0-46e6-bade-9366f14c4f46 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:53.381379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f02bc115-4cf0-46e6-bade-9366f14c4f46 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:53.381737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f02bc115-4cf0-46e6-bade-9366f14c4f46 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:53.382465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f02bc115-4cf0-46e6-bade-9366f14c4f46 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:53.382912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f02bc115-4cf0-46e6-bade-9366f14c4f46 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:53.383330 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f02bc115-4cf0-46e6-bade-9366f14c4f46 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:53.417191 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f02bc115-4cf0-46e6-bade-9366f14c4f46 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051718 Sep 08 14:03:53.417620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 809/1724] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:53 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:53.422837 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3d0170b6-bda5-43ee-8352-17d8836ba22b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:53.423444 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3d0170b6-bda5-43ee-8352-17d8836ba22b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:53.426978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0c3ac280-d0b2-4063-b203-e84f76b1d342 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:53.431775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d0170b6-bda5-43ee-8352-17d8836ba22b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:53.431775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d0170b6-bda5-43ee-8352-17d8836ba22b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:53.431775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d0170b6-bda5-43ee-8352-17d8836ba22b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:53.431775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d0170b6-bda5-43ee-8352-17d8836ba22b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:53.432148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d0170b6-bda5-43ee-8352-17d8836ba22b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:53.432148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3d0170b6-bda5-43ee-8352-17d8836ba22b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:53.441322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0c3ac280-d0b2-4063-b203-e84f76b1d342 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.191289 Sep 08 14:03:53.442296 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 916/1725] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:53 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:53.477223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4d8c727c-9454-4ce5-9066-5b36b5cad85e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:53.479576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4d8c727c-9454-4ce5-9066-5b36b5cad85e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:53.488212 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4d8c727c-9454-4ce5-9066-5b36b5cad85e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:53.488709 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4d8c727c-9454-4ce5-9066-5b36b5cad85e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:53.489148 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4d8c727c-9454-4ce5-9066-5b36b5cad85e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:53.489950 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4d8c727c-9454-4ce5-9066-5b36b5cad85e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:53.490444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4d8c727c-9454-4ce5-9066-5b36b5cad85e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:53.490878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4d8c727c-9454-4ce5-9066-5b36b5cad85e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:53.509517 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3d0170b6-bda5-43ee-8352-17d8836ba22b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:53.521015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3d0170b6-bda5-43ee-8352-17d8836ba22b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.102218 Sep 08 14:03:53.521612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 810/1726] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:53 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:53.535411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4d8c727c-9454-4ce5-9066-5b36b5cad85e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.065104 Sep 08 14:03:53.535889 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 917/1727] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:53 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:53.777382 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-71541fbf-50d9-4037-80a0-2f00118e1e11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:53.779508 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-71541fbf-50d9-4037-80a0-2f00118e1e11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:53.786514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71541fbf-50d9-4037-80a0-2f00118e1e11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:53.787001 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71541fbf-50d9-4037-80a0-2f00118e1e11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:53.787429 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71541fbf-50d9-4037-80a0-2f00118e1e11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:53.788133 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71541fbf-50d9-4037-80a0-2f00118e1e11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:53.788582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71541fbf-50d9-4037-80a0-2f00118e1e11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:53.789009 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71541fbf-50d9-4037-80a0-2f00118e1e11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:53.828578 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-71541fbf-50d9-4037-80a0-2f00118e1e11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055120 Sep 08 14:03:53.829176 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 811/1728] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:53 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:54.432513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e2856d8d-5397-40af-8c3e-27238c82d0ab tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:54.434791 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e2856d8d-5397-40af-8c3e-27238c82d0ab tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:54.441908 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e2856d8d-5397-40af-8c3e-27238c82d0ab tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:54.442425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e2856d8d-5397-40af-8c3e-27238c82d0ab tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:54.442869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e2856d8d-5397-40af-8c3e-27238c82d0ab tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:54.443582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e2856d8d-5397-40af-8c3e-27238c82d0ab tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:54.444056 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e2856d8d-5397-40af-8c3e-27238c82d0ab tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:54.444481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e2856d8d-5397-40af-8c3e-27238c82d0ab tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:54.476566 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a2b4ed15-2fb4-4fa6-ab98-b07093aff8a2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:54.477285 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a2b4ed15-2fb4-4fa6-ab98-b07093aff8a2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:54.485321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a2b4ed15-2fb4-4fa6-ab98-b07093aff8a2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:54.485321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a2b4ed15-2fb4-4fa6-ab98-b07093aff8a2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:54.485321 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a2b4ed15-2fb4-4fa6-ab98-b07093aff8a2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:54.486365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e2856d8d-5397-40af-8c3e-27238c82d0ab tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057124 Sep 08 14:03:54.486773 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a2b4ed15-2fb4-4fa6-ab98-b07093aff8a2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:54.486947 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 918/1729] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:54 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:54.487131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a2b4ed15-2fb4-4fa6-ab98-b07093aff8a2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:54.487478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a2b4ed15-2fb4-4fa6-ab98-b07093aff8a2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:54.548500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d51be4fe-84d7-47d2-b2fa-e4fe0025ae27 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:54.550659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d51be4fe-84d7-47d2-b2fa-e4fe0025ae27 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:54.556739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d51be4fe-84d7-47d2-b2fa-e4fe0025ae27 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:54.557225 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d51be4fe-84d7-47d2-b2fa-e4fe0025ae27 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:54.557674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d51be4fe-84d7-47d2-b2fa-e4fe0025ae27 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:54.558387 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d51be4fe-84d7-47d2-b2fa-e4fe0025ae27 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:54.558838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d51be4fe-84d7-47d2-b2fa-e4fe0025ae27 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:54.559260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d51be4fe-84d7-47d2-b2fa-e4fe0025ae27 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:54.590653 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d51be4fe-84d7-47d2-b2fa-e4fe0025ae27 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046266 Sep 08 14:03:54.595670 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 919/1730] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:54 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:54.596513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c4e04d34-0d58-4e2a-96b7-3ac6dd641d85 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:54.597144 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c4e04d34-0d58-4e2a-96b7-3ac6dd641d85 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:54.607470 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c4e04d34-0d58-4e2a-96b7-3ac6dd641d85 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:54.609508 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c4e04d34-0d58-4e2a-96b7-3ac6dd641d85 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:54.609508 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c4e04d34-0d58-4e2a-96b7-3ac6dd641d85 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:54.609508 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c4e04d34-0d58-4e2a-96b7-3ac6dd641d85 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:54.609508 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c4e04d34-0d58-4e2a-96b7-3ac6dd641d85 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:54.609955 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c4e04d34-0d58-4e2a-96b7-3ac6dd641d85 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:54.643256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a2b4ed15-2fb4-4fa6-ab98-b07093aff8a2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:54.655357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a2b4ed15-2fb4-4fa6-ab98-b07093aff8a2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.182584 Sep 08 14:03:54.655957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 812/1731] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:54 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:54.683581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c4e04d34-0d58-4e2a-96b7-3ac6dd641d85 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:54.692272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c4e04d34-0d58-4e2a-96b7-3ac6dd641d85 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.099478 Sep 08 14:03:54.693513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 920/1732] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:54 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:54.841063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ccbf604e-f4db-453b-8fb4-bfcf3fb03a22 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:54.843460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ccbf604e-f4db-453b-8fb4-bfcf3fb03a22 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:54.849840 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccbf604e-f4db-453b-8fb4-bfcf3fb03a22 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:54.850379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccbf604e-f4db-453b-8fb4-bfcf3fb03a22 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:54.850817 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccbf604e-f4db-453b-8fb4-bfcf3fb03a22 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:54.851524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccbf604e-f4db-453b-8fb4-bfcf3fb03a22 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:54.851983 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccbf604e-f4db-453b-8fb4-bfcf3fb03a22 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:54.852410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ccbf604e-f4db-453b-8fb4-bfcf3fb03a22 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:54.885628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ccbf604e-f4db-453b-8fb4-bfcf3fb03a22 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048548 Sep 08 14:03:54.886256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 813/1733] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:54 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:55.500114 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-529bc499-774e-443b-a564-f953ecedfda2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:55.503114 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-529bc499-774e-443b-a564-f953ecedfda2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:55.510354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-529bc499-774e-443b-a564-f953ecedfda2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:55.510880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-529bc499-774e-443b-a564-f953ecedfda2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:55.511356 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-529bc499-774e-443b-a564-f953ecedfda2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:55.512165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-529bc499-774e-443b-a564-f953ecedfda2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:55.512689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-529bc499-774e-443b-a564-f953ecedfda2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:55.513155 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-529bc499-774e-443b-a564-f953ecedfda2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:55.552773 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-529bc499-774e-443b-a564-f953ecedfda2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058716 Sep 08 14:03:55.553368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 921/1734] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:55 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:55.603470 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fc58d551-8b4a-4052-a645-5182344f3010 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:55.605699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fc58d551-8b4a-4052-a645-5182344f3010 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:55.613314 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc58d551-8b4a-4052-a645-5182344f3010 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:55.613809 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc58d551-8b4a-4052-a645-5182344f3010 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:55.614292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc58d551-8b4a-4052-a645-5182344f3010 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:55.615009 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc58d551-8b4a-4052-a645-5182344f3010 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:55.615463 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc58d551-8b4a-4052-a645-5182344f3010 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:55.615889 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fc58d551-8b4a-4052-a645-5182344f3010 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:55.647917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fc58d551-8b4a-4052-a645-5182344f3010 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048420 Sep 08 14:03:55.648487 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 814/1735] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:55 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:55.679426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2429f5c0-3e38-4b54-ba6e-3734fdbe62c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:55.680046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2429f5c0-3e38-4b54-ba6e-3734fdbe62c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:55.687576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2429f5c0-3e38-4b54-ba6e-3734fdbe62c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:55.688072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2429f5c0-3e38-4b54-ba6e-3734fdbe62c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:55.688514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2429f5c0-3e38-4b54-ba6e-3734fdbe62c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:55.689234 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2429f5c0-3e38-4b54-ba6e-3734fdbe62c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:55.689688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2429f5c0-3e38-4b54-ba6e-3734fdbe62c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:55.690115 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2429f5c0-3e38-4b54-ba6e-3734fdbe62c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:55.716461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2258b202-fedb-415d-9719-460605c41da3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:55.717083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2258b202-fedb-415d-9719-460605c41da3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:55.723529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2258b202-fedb-415d-9719-460605c41da3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:55.724021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2258b202-fedb-415d-9719-460605c41da3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:55.724466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2258b202-fedb-415d-9719-460605c41da3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:55.725178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2258b202-fedb-415d-9719-460605c41da3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:55.725639 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2258b202-fedb-415d-9719-460605c41da3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:55.726064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2258b202-fedb-415d-9719-460605c41da3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:55.759996 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:03:55.760256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:03:55 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 (ip 10.222.0.25) !!! Sep 08 14:03:55.800281 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-2258b202-fedb-415d-9719-460605c41da3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:55.808386 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2258b202-fedb-415d-9719-460605c41da3 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.095781 Sep 08 14:03:55.810625 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 815/1736] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:55 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:03:55.831546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2429f5c0-3e38-4b54-ba6e-3734fdbe62c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:55.839517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2429f5c0-3e38-4b54-ba6e-3734fdbe62c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.163964 Sep 08 14:03:55.840169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 922/1737] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:55 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:55.898598 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6e5f10d8-59bc-4082-af9f-2c4d68acc583 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:55.900785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6e5f10d8-59bc-4082-af9f-2c4d68acc583 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:55.902838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:55.908456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6e5f10d8-59bc-4082-af9f-2c4d68acc583 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:55.908955 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6e5f10d8-59bc-4082-af9f-2c4d68acc583 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:55.909420 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6e5f10d8-59bc-4082-af9f-2c4d68acc583 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:55.910114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6e5f10d8-59bc-4082-af9f-2c4d68acc583 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:55.910608 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6e5f10d8-59bc-4082-af9f-2c4d68acc583 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:55.924302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6e5f10d8-59bc-4082-af9f-2c4d68acc583 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:55.943879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6e5f10d8-59bc-4082-af9f-2c4d68acc583 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049221 Sep 08 14:03:55.944492 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 816/1738] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:55 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:03:56.566739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-eed87f44-55e3-4cae-a6bf-de5cf0455d5b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:56.568886 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-eed87f44-55e3-4cae-a6bf-de5cf0455d5b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:56.575548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eed87f44-55e3-4cae-a6bf-de5cf0455d5b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:56.576037 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eed87f44-55e3-4cae-a6bf-de5cf0455d5b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:56.576515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eed87f44-55e3-4cae-a6bf-de5cf0455d5b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:56.577299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eed87f44-55e3-4cae-a6bf-de5cf0455d5b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:56.577755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eed87f44-55e3-4cae-a6bf-de5cf0455d5b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:56.578198 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eed87f44-55e3-4cae-a6bf-de5cf0455d5b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:56.612689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-eed87f44-55e3-4cae-a6bf-de5cf0455d5b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050124 Sep 08 14:03:56.613357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 923/1739] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:56 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:03:56.661455 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d68acd58-c93c-48aa-9eca-c0891eb1c23c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:56.663632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d68acd58-c93c-48aa-9eca-c0891eb1c23c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:56.670256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d68acd58-c93c-48aa-9eca-c0891eb1c23c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:56.670735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d68acd58-c93c-48aa-9eca-c0891eb1c23c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:56.671164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d68acd58-c93c-48aa-9eca-c0891eb1c23c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:56.671846 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d68acd58-c93c-48aa-9eca-c0891eb1c23c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:56.672288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d68acd58-c93c-48aa-9eca-c0891eb1c23c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:56.672752 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d68acd58-c93c-48aa-9eca-c0891eb1c23c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:56.704170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d68acd58-c93c-48aa-9eca-c0891eb1c23c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047134 Sep 08 14:03:56.704734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 817/1740] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:56 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:56.832807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1eb1d9c2-8b90-4a61-9a38-7c1209492a89 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:56.833351 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1eb1d9c2-8b90-4a61-9a38-7c1209492a89 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:56.839847 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1eb1d9c2-8b90-4a61-9a38-7c1209492a89 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:56.840376 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1eb1d9c2-8b90-4a61-9a38-7c1209492a89 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:56.841039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1eb1d9c2-8b90-4a61-9a38-7c1209492a89 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:56.842391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1eb1d9c2-8b90-4a61-9a38-7c1209492a89 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:56.842940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1eb1d9c2-8b90-4a61-9a38-7c1209492a89 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:56.843385 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1eb1d9c2-8b90-4a61-9a38-7c1209492a89 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:56.864317 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5d4a6347-6d1e-4fdc-8cc7-6cdbe3a01541 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:56.865343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5d4a6347-6d1e-4fdc-8cc7-6cdbe3a01541 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:56.872957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d4a6347-6d1e-4fdc-8cc7-6cdbe3a01541 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:56.874371 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d4a6347-6d1e-4fdc-8cc7-6cdbe3a01541 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:56.875273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d4a6347-6d1e-4fdc-8cc7-6cdbe3a01541 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:56.876176 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d4a6347-6d1e-4fdc-8cc7-6cdbe3a01541 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:56.876621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d4a6347-6d1e-4fdc-8cc7-6cdbe3a01541 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:56.877136 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d4a6347-6d1e-4fdc-8cc7-6cdbe3a01541 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:56.924954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1eb1d9c2-8b90-4a61-9a38-7c1209492a89 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:56.934402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1eb1d9c2-8b90-4a61-9a38-7c1209492a89 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.105902 Sep 08 14:03:56.935475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 924/1741] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:56 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:56.956896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-44592453-5a41-47b1-a866-6b5274c4de8d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:56.959928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-44592453-5a41-47b1-a866-6b5274c4de8d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:56.966867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44592453-5a41-47b1-a866-6b5274c4de8d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:56.967835 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44592453-5a41-47b1-a866-6b5274c4de8d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:56.968743 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44592453-5a41-47b1-a866-6b5274c4de8d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:56.970210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44592453-5a41-47b1-a866-6b5274c4de8d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:56.970930 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44592453-5a41-47b1-a866-6b5274c4de8d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:56.972896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44592453-5a41-47b1-a866-6b5274c4de8d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:57.006389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-44592453-5a41-47b1-a866-6b5274c4de8d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053527 Sep 08 14:03:57.006964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 925/1742] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:56 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:03:57.036765 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5d4a6347-6d1e-4fdc-8cc7-6cdbe3a01541 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:57.050481 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5d4a6347-6d1e-4fdc-8cc7-6cdbe3a01541 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.190715 Sep 08 14:03:57.051406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 818/1743] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:56 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:57.625558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e1629e80-a531-4e4e-8b49-3af778f4f3e8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:57.627858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e1629e80-a531-4e4e-8b49-3af778f4f3e8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:57.634793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1629e80-a531-4e4e-8b49-3af778f4f3e8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:57.635205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1629e80-a531-4e4e-8b49-3af778f4f3e8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:57.635649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1629e80-a531-4e4e-8b49-3af778f4f3e8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:57.636436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1629e80-a531-4e4e-8b49-3af778f4f3e8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:57.636929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1629e80-a531-4e4e-8b49-3af778f4f3e8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:57.637360 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e1629e80-a531-4e4e-8b49-3af778f4f3e8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:57.640902 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:57.670451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e1629e80-a531-4e4e-8b49-3af778f4f3e8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048956 Sep 08 14:03:57.671083 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 926/1744] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:57 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:57.716753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6c2a741e-6d5e-4b11-93b7-502af4efa713 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:57.719740 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6c2a741e-6d5e-4b11-93b7-502af4efa713 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:57.726534 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c2a741e-6d5e-4b11-93b7-502af4efa713 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:57.727031 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c2a741e-6d5e-4b11-93b7-502af4efa713 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:57.727470 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c2a741e-6d5e-4b11-93b7-502af4efa713 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:57.728182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c2a741e-6d5e-4b11-93b7-502af4efa713 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:57.728638 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c2a741e-6d5e-4b11-93b7-502af4efa713 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:57.729068 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c2a741e-6d5e-4b11-93b7-502af4efa713 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:57.760726 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6c2a741e-6d5e-4b11-93b7-502af4efa713 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048110 Sep 08 14:03:57.761320 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 819/1745] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:57 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:57.958943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0cafb713-106d-40e4-83e1-2596fbe7c33d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:57.959562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0cafb713-106d-40e4-83e1-2596fbe7c33d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:57.966114 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0cafb713-106d-40e4-83e1-2596fbe7c33d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:57.966635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0cafb713-106d-40e4-83e1-2596fbe7c33d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:57.967081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0cafb713-106d-40e4-83e1-2596fbe7c33d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:57.968386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0cafb713-106d-40e4-83e1-2596fbe7c33d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:57.969078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0cafb713-106d-40e4-83e1-2596fbe7c33d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:57.969588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0cafb713-106d-40e4-83e1-2596fbe7c33d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:58.019724 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4d394e7d-60e2-4eda-99bb-286206361dc2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:58.021978 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4d394e7d-60e2-4eda-99bb-286206361dc2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:58.028969 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d394e7d-60e2-4eda-99bb-286206361dc2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:58.029464 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d394e7d-60e2-4eda-99bb-286206361dc2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:58.029901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d394e7d-60e2-4eda-99bb-286206361dc2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:58.030654 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d394e7d-60e2-4eda-99bb-286206361dc2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:58.031114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d394e7d-60e2-4eda-99bb-286206361dc2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:58.031539 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d394e7d-60e2-4eda-99bb-286206361dc2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:58.037777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0cafb713-106d-40e4-83e1-2596fbe7c33d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:58.047003 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0cafb713-106d-40e4-83e1-2596fbe7c33d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.092129 Sep 08 14:03:58.049591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 927/1746] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:57 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:58.064334 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4d394e7d-60e2-4eda-99bb-286206361dc2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048539 Sep 08 14:03:58.064925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 820/1747] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:58 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:03:58.081637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7105c5ea-9049-46b9-9c2d-bee219d724e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:58.082284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7105c5ea-9049-46b9-9c2d-bee219d724e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:58.088330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7105c5ea-9049-46b9-9c2d-bee219d724e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:58.088839 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7105c5ea-9049-46b9-9c2d-bee219d724e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:58.089306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7105c5ea-9049-46b9-9c2d-bee219d724e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:58.090012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7105c5ea-9049-46b9-9c2d-bee219d724e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:58.090504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7105c5ea-9049-46b9-9c2d-bee219d724e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:58.091271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7105c5ea-9049-46b9-9c2d-bee219d724e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:58.228563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7105c5ea-9049-46b9-9c2d-bee219d724e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:58.237758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7105c5ea-9049-46b9-9c2d-bee219d724e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.160643 Sep 08 14:03:58.238383 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 928/1748] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:58 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:58.683346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3b8838aa-c619-40e4-a220-d45e505e3c8c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:58.685483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3b8838aa-c619-40e4-a220-d45e505e3c8c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:58.692199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b8838aa-c619-40e4-a220-d45e505e3c8c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:58.692680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b8838aa-c619-40e4-a220-d45e505e3c8c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:58.693111 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b8838aa-c619-40e4-a220-d45e505e3c8c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:58.693802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b8838aa-c619-40e4-a220-d45e505e3c8c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:58.694277 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b8838aa-c619-40e4-a220-d45e505e3c8c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:58.694698 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3b8838aa-c619-40e4-a220-d45e505e3c8c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:58.723665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3b8838aa-c619-40e4-a220-d45e505e3c8c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044280 Sep 08 14:03:58.724346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 821/1749] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:58 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:58.774053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-50e22fdf-0eb6-46dc-b718-35bff43254f1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:58.776223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-50e22fdf-0eb6-46dc-b718-35bff43254f1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:58.783220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50e22fdf-0eb6-46dc-b718-35bff43254f1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:58.783704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50e22fdf-0eb6-46dc-b718-35bff43254f1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:58.784142 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50e22fdf-0eb6-46dc-b718-35bff43254f1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:58.784829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50e22fdf-0eb6-46dc-b718-35bff43254f1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:58.785283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50e22fdf-0eb6-46dc-b718-35bff43254f1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:58.785708 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50e22fdf-0eb6-46dc-b718-35bff43254f1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:58.815174 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-50e22fdf-0eb6-46dc-b718-35bff43254f1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045138 Sep 08 14:03:58.815718 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 929/1750] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:58 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:59.071400 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ea3d1fdd-49ca-430e-b87e-3237a41abd21 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:59.071948 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ea3d1fdd-49ca-430e-b87e-3237a41abd21 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:59.076587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-900e4d82-4959-4958-b5d7-df994a80fd67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:59.079966 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ea3d1fdd-49ca-430e-b87e-3237a41abd21 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:59.080110 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-900e4d82-4959-4958-b5d7-df994a80fd67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:59.080225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ea3d1fdd-49ca-430e-b87e-3237a41abd21 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:59.080225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ea3d1fdd-49ca-430e-b87e-3237a41abd21 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:59.080225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ea3d1fdd-49ca-430e-b87e-3237a41abd21 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:59.080693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ea3d1fdd-49ca-430e-b87e-3237a41abd21 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:59.081121 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ea3d1fdd-49ca-430e-b87e-3237a41abd21 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:59.084803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-900e4d82-4959-4958-b5d7-df994a80fd67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:59.085245 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:59.085373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-900e4d82-4959-4958-b5d7-df994a80fd67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:59.085720 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-900e4d82-4959-4958-b5d7-df994a80fd67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:59.086447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-900e4d82-4959-4958-b5d7-df994a80fd67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:59.086907 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-900e4d82-4959-4958-b5d7-df994a80fd67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:59.087336 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-900e4d82-4959-4958-b5d7-df994a80fd67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:59.119853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-900e4d82-4959-4958-b5d7-df994a80fd67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047320 Sep 08 14:03:59.120706 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 930/1751] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:03:59 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:03:59.150735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ea3d1fdd-49ca-430e-b87e-3237a41abd21 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:59.160028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ea3d1fdd-49ca-430e-b87e-3237a41abd21 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.092521 Sep 08 14:03:59.160612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 822/1752] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:59 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:59.260966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c9390958-40e2-4580-86b6-eb9a097fdbe3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:59.261565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c9390958-40e2-4580-86b6-eb9a097fdbe3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:59.267987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c9390958-40e2-4580-86b6-eb9a097fdbe3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:59.268464 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c9390958-40e2-4580-86b6-eb9a097fdbe3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:59.268897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c9390958-40e2-4580-86b6-eb9a097fdbe3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:59.269574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c9390958-40e2-4580-86b6-eb9a097fdbe3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:59.270021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c9390958-40e2-4580-86b6-eb9a097fdbe3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:59.270471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c9390958-40e2-4580-86b6-eb9a097fdbe3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:59.404116 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c9390958-40e2-4580-86b6-eb9a097fdbe3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:03:59.412674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c9390958-40e2-4580-86b6-eb9a097fdbe3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.155682 Sep 08 14:03:59.413242 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 931/1753] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:03:59 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:03:59.737804 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ed2a5f4e-7e1a-44b4-b002-094158576570 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:59.739978 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ed2a5f4e-7e1a-44b4-b002-094158576570 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:59.746699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed2a5f4e-7e1a-44b4-b002-094158576570 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:59.747602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed2a5f4e-7e1a-44b4-b002-094158576570 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:59.747602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed2a5f4e-7e1a-44b4-b002-094158576570 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:59.748320 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed2a5f4e-7e1a-44b4-b002-094158576570 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:59.749262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed2a5f4e-7e1a-44b4-b002-094158576570 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:59.749684 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed2a5f4e-7e1a-44b4-b002-094158576570 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:59.783048 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ed2a5f4e-7e1a-44b4-b002-094158576570 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049161 Sep 08 14:03:59.783630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 823/1754] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:59 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:03:59.828449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6305f531-f151-431e-a2c7-7b0a0c47d42a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:03:59.830639 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6305f531-f151-431e-a2c7-7b0a0c47d42a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:03:59.836816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6305f531-f151-431e-a2c7-7b0a0c47d42a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:59.837290 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6305f531-f151-431e-a2c7-7b0a0c47d42a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:59.837718 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6305f531-f151-431e-a2c7-7b0a0c47d42a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:59.838436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6305f531-f151-431e-a2c7-7b0a0c47d42a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:03:59.838881 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6305f531-f151-431e-a2c7-7b0a0c47d42a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:03:59.839297 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6305f531-f151-431e-a2c7-7b0a0c47d42a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:03:59.868569 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6305f531-f151-431e-a2c7-7b0a0c47d42a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044760 Sep 08 14:03:59.869123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 932/1755] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:03:59 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:00.133889 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fee481c7-16fe-4390-9f12-dfb281f0d045 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:00.137817 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fee481c7-16fe-4390-9f12-dfb281f0d045 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:00.148167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fee481c7-16fe-4390-9f12-dfb281f0d045 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:00.148735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fee481c7-16fe-4390-9f12-dfb281f0d045 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:00.149170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fee481c7-16fe-4390-9f12-dfb281f0d045 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:00.149922 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fee481c7-16fe-4390-9f12-dfb281f0d045 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:00.150402 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fee481c7-16fe-4390-9f12-dfb281f0d045 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:00.150829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fee481c7-16fe-4390-9f12-dfb281f0d045 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:00.183969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d85bf30f-fdbe-4e03-8d55-2c3cbd35e91c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:00.184538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d85bf30f-fdbe-4e03-8d55-2c3cbd35e91c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:00.187871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fee481c7-16fe-4390-9f12-dfb281f0d045 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058437 Sep 08 14:04:00.188988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 824/1756] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:00 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:00.193311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d85bf30f-fdbe-4e03-8d55-2c3cbd35e91c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:00.193913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d85bf30f-fdbe-4e03-8d55-2c3cbd35e91c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:00.194629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d85bf30f-fdbe-4e03-8d55-2c3cbd35e91c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:00.195453 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d85bf30f-fdbe-4e03-8d55-2c3cbd35e91c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:00.196201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d85bf30f-fdbe-4e03-8d55-2c3cbd35e91c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:00.196722 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d85bf30f-fdbe-4e03-8d55-2c3cbd35e91c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:00.265966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d85bf30f-fdbe-4e03-8d55-2c3cbd35e91c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:00.273917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d85bf30f-fdbe-4e03-8d55-2c3cbd35e91c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.094078 Sep 08 14:04:00.274524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 933/1757] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:00 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:00.440480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-84f6d60f-568b-4495-9b6e-9736e57d3ff2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:00.440480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-84f6d60f-568b-4495-9b6e-9736e57d3ff2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:00.442921 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84f6d60f-568b-4495-9b6e-9736e57d3ff2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:00.443489 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84f6d60f-568b-4495-9b6e-9736e57d3ff2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:00.444034 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84f6d60f-568b-4495-9b6e-9736e57d3ff2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:00.444853 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84f6d60f-568b-4495-9b6e-9736e57d3ff2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:00.445450 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84f6d60f-568b-4495-9b6e-9736e57d3ff2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:00.445975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-84f6d60f-568b-4495-9b6e-9736e57d3ff2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:00.587260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-84f6d60f-568b-4495-9b6e-9736e57d3ff2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:00.595888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-84f6d60f-568b-4495-9b6e-9736e57d3ff2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.164552 Sep 08 14:04:00.596378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 825/1758] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:00 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:00.796358 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-88d20e17-f51d-468f-99ea-758287dc2fc6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:00.798451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-88d20e17-f51d-468f-99ea-758287dc2fc6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:00.804535 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-88d20e17-f51d-468f-99ea-758287dc2fc6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:00.805030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-88d20e17-f51d-468f-99ea-758287dc2fc6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:00.805465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-88d20e17-f51d-468f-99ea-758287dc2fc6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:00.806187 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-88d20e17-f51d-468f-99ea-758287dc2fc6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:00.806643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-88d20e17-f51d-468f-99ea-758287dc2fc6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:00.807069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-88d20e17-f51d-468f-99ea-758287dc2fc6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:00.838740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-88d20e17-f51d-468f-99ea-758287dc2fc6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046740 Sep 08 14:04:00.839435 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 934/1759] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:00 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:00.881010 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5a976c57-aef2-44ef-8ea0-a87e913d9d6f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:00.883348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5a976c57-aef2-44ef-8ea0-a87e913d9d6f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:00.889974 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5a976c57-aef2-44ef-8ea0-a87e913d9d6f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:00.890563 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5a976c57-aef2-44ef-8ea0-a87e913d9d6f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:00.891002 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5a976c57-aef2-44ef-8ea0-a87e913d9d6f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:00.891732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5a976c57-aef2-44ef-8ea0-a87e913d9d6f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:00.892190 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5a976c57-aef2-44ef-8ea0-a87e913d9d6f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:00.892620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5a976c57-aef2-44ef-8ea0-a87e913d9d6f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:00.928635 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5a976c57-aef2-44ef-8ea0-a87e913d9d6f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051560 Sep 08 14:04:00.929211 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 826/1760] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:00 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:01.203293 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f89788ea-6f70-4a0b-a7ce-ff414de420ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:01.205476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f89788ea-6f70-4a0b-a7ce-ff414de420ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:01.211966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f89788ea-6f70-4a0b-a7ce-ff414de420ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:01.212456 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f89788ea-6f70-4a0b-a7ce-ff414de420ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:01.212896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f89788ea-6f70-4a0b-a7ce-ff414de420ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:01.213595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f89788ea-6f70-4a0b-a7ce-ff414de420ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:01.214053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f89788ea-6f70-4a0b-a7ce-ff414de420ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:01.214570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f89788ea-6f70-4a0b-a7ce-ff414de420ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:01.244723 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f89788ea-6f70-4a0b-a7ce-ff414de420ea tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046283 Sep 08 14:04:01.245553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 935/1761] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:01 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:01.297465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3e797d14-1008-4ce7-a91e-432f1c58a2b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:01.298063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3e797d14-1008-4ce7-a91e-432f1c58a2b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:01.304835 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e797d14-1008-4ce7-a91e-432f1c58a2b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:01.305319 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e797d14-1008-4ce7-a91e-432f1c58a2b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:01.305752 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e797d14-1008-4ce7-a91e-432f1c58a2b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:01.306472 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e797d14-1008-4ce7-a91e-432f1c58a2b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:01.306926 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e797d14-1008-4ce7-a91e-432f1c58a2b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:01.307348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e797d14-1008-4ce7-a91e-432f1c58a2b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:01.370079 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3e797d14-1008-4ce7-a91e-432f1c58a2b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:01.379063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3e797d14-1008-4ce7-a91e-432f1c58a2b5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.085458 Sep 08 14:04:01.379643 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 827/1762] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:01 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:01.620147 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e62ad64d-2440-4b50-bc02-52aa1ce2b963 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:01.627709 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e62ad64d-2440-4b50-bc02-52aa1ce2b963 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:01.630523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e62ad64d-2440-4b50-bc02-52aa1ce2b963 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:01.631151 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e62ad64d-2440-4b50-bc02-52aa1ce2b963 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:01.631151 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e62ad64d-2440-4b50-bc02-52aa1ce2b963 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:01.632233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e62ad64d-2440-4b50-bc02-52aa1ce2b963 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:01.632708 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e62ad64d-2440-4b50-bc02-52aa1ce2b963 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:01.633139 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e62ad64d-2440-4b50-bc02-52aa1ce2b963 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:01.780937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e62ad64d-2440-4b50-bc02-52aa1ce2b963 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:01.789238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e62ad64d-2440-4b50-bc02-52aa1ce2b963 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.173740 Sep 08 14:04:01.789852 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 936/1763] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:01 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:01.852538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-727de4a9-7248-4e63-aa8d-b63d96275825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:01.854796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-727de4a9-7248-4e63-aa8d-b63d96275825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:01.864998 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-727de4a9-7248-4e63-aa8d-b63d96275825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:01.865817 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-727de4a9-7248-4e63-aa8d-b63d96275825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:01.866623 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-727de4a9-7248-4e63-aa8d-b63d96275825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:01.867890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-727de4a9-7248-4e63-aa8d-b63d96275825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:01.868758 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-727de4a9-7248-4e63-aa8d-b63d96275825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:01.869711 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-727de4a9-7248-4e63-aa8d-b63d96275825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:01.903743 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-727de4a9-7248-4e63-aa8d-b63d96275825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055315 Sep 08 14:04:01.904405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 828/1764] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:01 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:01.941487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-849139ef-05aa-491c-8a4a-d62223770515 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:01.947999 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-849139ef-05aa-491c-8a4a-d62223770515 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:01.950259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-849139ef-05aa-491c-8a4a-d62223770515 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:01.950747 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-849139ef-05aa-491c-8a4a-d62223770515 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:01.951188 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-849139ef-05aa-491c-8a4a-d62223770515 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:01.952664 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-849139ef-05aa-491c-8a4a-d62223770515 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:01.953159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-849139ef-05aa-491c-8a4a-d62223770515 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:01.953586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-849139ef-05aa-491c-8a4a-d62223770515 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:01.983564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-849139ef-05aa-491c-8a4a-d62223770515 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045824 Sep 08 14:04:01.983972 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 937/1765] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:01 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:02.258560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-73c48d9f-cfe4-4da6-9049-fdae5c5c486b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:02.260638 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-73c48d9f-cfe4-4da6-9049-fdae5c5c486b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:02.267128 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73c48d9f-cfe4-4da6-9049-fdae5c5c486b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:02.267614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73c48d9f-cfe4-4da6-9049-fdae5c5c486b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:02.268042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73c48d9f-cfe4-4da6-9049-fdae5c5c486b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:02.268734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73c48d9f-cfe4-4da6-9049-fdae5c5c486b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:02.269178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73c48d9f-cfe4-4da6-9049-fdae5c5c486b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:02.269592 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73c48d9f-cfe4-4da6-9049-fdae5c5c486b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:02.302102 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-73c48d9f-cfe4-4da6-9049-fdae5c5c486b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047677 Sep 08 14:04:02.302698 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 829/1766] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:02 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:02.402660 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a1c06504-0d92-4c37-b488-d6fefd3749a8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:02.403268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a1c06504-0d92-4c37-b488-d6fefd3749a8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:02.409589 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1c06504-0d92-4c37-b488-d6fefd3749a8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:02.410087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1c06504-0d92-4c37-b488-d6fefd3749a8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:02.410578 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1c06504-0d92-4c37-b488-d6fefd3749a8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:02.411285 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1c06504-0d92-4c37-b488-d6fefd3749a8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:02.411741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1c06504-0d92-4c37-b488-d6fefd3749a8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:02.412174 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1c06504-0d92-4c37-b488-d6fefd3749a8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:02.474375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a1c06504-0d92-4c37-b488-d6fefd3749a8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:02.482394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a1c06504-0d92-4c37-b488-d6fefd3749a8 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.083593 Sep 08 14:04:02.483499 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 938/1767] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:02 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:02.813079 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-429a30ae-c2e6-4262-aeb6-85467bc8e866 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:02.813690 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-429a30ae-c2e6-4262-aeb6-85467bc8e866 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:02.820343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-429a30ae-c2e6-4262-aeb6-85467bc8e866 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:02.820829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-429a30ae-c2e6-4262-aeb6-85467bc8e866 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:02.821269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-429a30ae-c2e6-4262-aeb6-85467bc8e866 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:02.821972 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-429a30ae-c2e6-4262-aeb6-85467bc8e866 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:02.822463 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-429a30ae-c2e6-4262-aeb6-85467bc8e866 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:02.822891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-429a30ae-c2e6-4262-aeb6-85467bc8e866 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:02.916188 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ca54182f-e3d2-4c22-817c-53d48cf4129f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:02.918660 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ca54182f-e3d2-4c22-817c-53d48cf4129f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:02.925108 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca54182f-e3d2-4c22-817c-53d48cf4129f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:02.925596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca54182f-e3d2-4c22-817c-53d48cf4129f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:02.926028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca54182f-e3d2-4c22-817c-53d48cf4129f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:02.926782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca54182f-e3d2-4c22-817c-53d48cf4129f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:02.927229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca54182f-e3d2-4c22-817c-53d48cf4129f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:02.927651 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca54182f-e3d2-4c22-817c-53d48cf4129f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:02.961186 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ca54182f-e3d2-4c22-817c-53d48cf4129f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049127 Sep 08 14:04:02.962192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 939/1768] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:02 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:02.971562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-429a30ae-c2e6-4262-aeb6-85467bc8e866 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:02.979457 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-429a30ae-c2e6-4262-aeb6-85467bc8e866 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.170245 Sep 08 14:04:02.980051 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 830/1769] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:02 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:02.996163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-819f43e7-2664-4289-8e31-f1849bb39f9d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:02.998410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-819f43e7-2664-4289-8e31-f1849bb39f9d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:03.000329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:03.005099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-819f43e7-2664-4289-8e31-f1849bb39f9d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:03.005589 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-819f43e7-2664-4289-8e31-f1849bb39f9d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:03.006029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-819f43e7-2664-4289-8e31-f1849bb39f9d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:03.006763 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-819f43e7-2664-4289-8e31-f1849bb39f9d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:03.007222 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-819f43e7-2664-4289-8e31-f1849bb39f9d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:03.007647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-819f43e7-2664-4289-8e31-f1849bb39f9d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:03.040404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-819f43e7-2664-4289-8e31-f1849bb39f9d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048413 Sep 08 14:04:03.040971 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 940/1770] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:02 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:03.315811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-767aa1ce-aef7-4d68-9f27-74b6d53f4868 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:03.317995 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-767aa1ce-aef7-4d68-9f27-74b6d53f4868 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:03.324173 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-767aa1ce-aef7-4d68-9f27-74b6d53f4868 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:03.324657 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-767aa1ce-aef7-4d68-9f27-74b6d53f4868 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:03.325092 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-767aa1ce-aef7-4d68-9f27-74b6d53f4868 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:03.325788 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-767aa1ce-aef7-4d68-9f27-74b6d53f4868 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:03.326902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-767aa1ce-aef7-4d68-9f27-74b6d53f4868 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:03.327334 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-767aa1ce-aef7-4d68-9f27-74b6d53f4868 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:03.357133 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-767aa1ce-aef7-4d68-9f27-74b6d53f4868 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.045325 Sep 08 14:04:03.357754 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 831/1771] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:03 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:03.508054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-155aeb6a-3319-422f-9b4b-e6cebc738328 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:03.508568 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-155aeb6a-3319-422f-9b4b-e6cebc738328 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:03.516188 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-155aeb6a-3319-422f-9b4b-e6cebc738328 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:03.516775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-155aeb6a-3319-422f-9b4b-e6cebc738328 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:03.517217 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-155aeb6a-3319-422f-9b4b-e6cebc738328 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:03.517921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-155aeb6a-3319-422f-9b4b-e6cebc738328 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:03.518398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-155aeb6a-3319-422f-9b4b-e6cebc738328 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:03.518821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-155aeb6a-3319-422f-9b4b-e6cebc738328 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:03.559431 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:04:03.559811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:04:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 (ip 10.222.0.25) !!! Sep 08 14:04:03.586522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-155aeb6a-3319-422f-9b4b-e6cebc738328 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:03.594540 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-155aeb6a-3319-422f-9b4b-e6cebc738328 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.091704 Sep 08 14:04:03.595111 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 941/1772] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:03 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:03.974408 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fd4edd01-ce47-410c-b2a5-a548899400c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:03.976570 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fd4edd01-ce47-410c-b2a5-a548899400c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:03.983963 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fd4edd01-ce47-410c-b2a5-a548899400c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:03.984446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fd4edd01-ce47-410c-b2a5-a548899400c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:03.984877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fd4edd01-ce47-410c-b2a5-a548899400c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:03.985584 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fd4edd01-ce47-410c-b2a5-a548899400c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:03.986030 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fd4edd01-ce47-410c-b2a5-a548899400c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:03.986482 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fd4edd01-ce47-410c-b2a5-a548899400c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:04.006454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1d602a20-fa6a-450c-8863-c22f8878c07e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:04.007139 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1d602a20-fa6a-450c-8863-c22f8878c07e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:04.013932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d602a20-fa6a-450c-8863-c22f8878c07e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:04.014423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d602a20-fa6a-450c-8863-c22f8878c07e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:04.015512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d602a20-fa6a-450c-8863-c22f8878c07e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:04.015512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d602a20-fa6a-450c-8863-c22f8878c07e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:04.015978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d602a20-fa6a-450c-8863-c22f8878c07e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:04.016399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d602a20-fa6a-450c-8863-c22f8878c07e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:04.019981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fd4edd01-ce47-410c-b2a5-a548899400c1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049634 Sep 08 14:04:04.020566 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 832/1773] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:03 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:04.053903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c958bf41-1fc6-43c9-8aa4-d2a39e458ce1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:04.056143 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c958bf41-1fc6-43c9-8aa4-d2a39e458ce1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:04.063049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c958bf41-1fc6-43c9-8aa4-d2a39e458ce1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:04.063543 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c958bf41-1fc6-43c9-8aa4-d2a39e458ce1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:04.063976 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c958bf41-1fc6-43c9-8aa4-d2a39e458ce1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:04.064700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c958bf41-1fc6-43c9-8aa4-d2a39e458ce1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:04.065147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c958bf41-1fc6-43c9-8aa4-d2a39e458ce1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:04.065580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c958bf41-1fc6-43c9-8aa4-d2a39e458ce1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:04.098064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c958bf41-1fc6-43c9-8aa4-d2a39e458ce1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048016 Sep 08 14:04:04.098680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 833/1774] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:04 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:04.161433 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1d602a20-fa6a-450c-8863-c22f8878c07e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:04.163778 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:04.170093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1d602a20-fa6a-450c-8863-c22f8878c07e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.167498 Sep 08 14:04:04.170883 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 942/1775] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:04 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:04.369313 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9a3ff335-374d-49c0-9109-bf69cff835a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:04.371792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9a3ff335-374d-49c0-9109-bf69cff835a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:04.378837 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9a3ff335-374d-49c0-9109-bf69cff835a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:04.379319 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9a3ff335-374d-49c0-9109-bf69cff835a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:04.379742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9a3ff335-374d-49c0-9109-bf69cff835a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:04.380435 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9a3ff335-374d-49c0-9109-bf69cff835a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:04.380877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9a3ff335-374d-49c0-9109-bf69cff835a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:04.381293 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9a3ff335-374d-49c0-9109-bf69cff835a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:04.413555 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9a3ff335-374d-49c0-9109-bf69cff835a1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048339 Sep 08 14:04:04.414164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 834/1776] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:04 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:04.620038 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-37ae5a0a-0a1f-4e2c-a261-d8615e532680 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:04.620539 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-37ae5a0a-0a1f-4e2c-a261-d8615e532680 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:04.627585 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-37ae5a0a-0a1f-4e2c-a261-d8615e532680 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:04.628069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-37ae5a0a-0a1f-4e2c-a261-d8615e532680 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:04.628502 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-37ae5a0a-0a1f-4e2c-a261-d8615e532680 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:04.629294 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-37ae5a0a-0a1f-4e2c-a261-d8615e532680 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:04.629820 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-37ae5a0a-0a1f-4e2c-a261-d8615e532680 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:04.630286 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-37ae5a0a-0a1f-4e2c-a261-d8615e532680 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:04.693965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-37ae5a0a-0a1f-4e2c-a261-d8615e532680 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:04.701997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-37ae5a0a-0a1f-4e2c-a261-d8615e532680 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.086817 Sep 08 14:04:04.702618 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 943/1777] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:04 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:05.032703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a0320810-8b69-418a-9ed3-e859e3eeea89 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:05.034873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a0320810-8b69-418a-9ed3-e859e3eeea89 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:05.040957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0320810-8b69-418a-9ed3-e859e3eeea89 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:05.041442 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0320810-8b69-418a-9ed3-e859e3eeea89 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:05.041905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0320810-8b69-418a-9ed3-e859e3eeea89 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:05.042616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0320810-8b69-418a-9ed3-e859e3eeea89 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:05.043062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0320810-8b69-418a-9ed3-e859e3eeea89 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:05.043478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0320810-8b69-418a-9ed3-e859e3eeea89 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:05.073726 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a0320810-8b69-418a-9ed3-e859e3eeea89 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045461 Sep 08 14:04:05.074311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 835/1778] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:05 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:05.111375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-35037ca8-3b76-4acd-b2a8-d035ef8d6f22 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:05.113544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-35037ca8-3b76-4acd-b2a8-d035ef8d6f22 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:05.120410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-35037ca8-3b76-4acd-b2a8-d035ef8d6f22 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:05.120898 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-35037ca8-3b76-4acd-b2a8-d035ef8d6f22 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:05.121340 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-35037ca8-3b76-4acd-b2a8-d035ef8d6f22 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:05.122078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-35037ca8-3b76-4acd-b2a8-d035ef8d6f22 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:05.122561 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-35037ca8-3b76-4acd-b2a8-d035ef8d6f22 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:05.123015 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-35037ca8-3b76-4acd-b2a8-d035ef8d6f22 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:05.153659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-35037ca8-3b76-4acd-b2a8-d035ef8d6f22 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046993 Sep 08 14:04:05.154938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 944/1779] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:05 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:05.199096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-015c9704-39e3-497a-8280-f49533ca3610 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:05.199731 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-015c9704-39e3-497a-8280-f49533ca3610 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:05.206332 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-015c9704-39e3-497a-8280-f49533ca3610 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:05.206807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-015c9704-39e3-497a-8280-f49533ca3610 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:05.207232 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-015c9704-39e3-497a-8280-f49533ca3610 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:05.207906 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-015c9704-39e3-497a-8280-f49533ca3610 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:05.208346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-015c9704-39e3-497a-8280-f49533ca3610 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:05.208760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-015c9704-39e3-497a-8280-f49533ca3610 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:05.350254 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-015c9704-39e3-497a-8280-f49533ca3610 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:05.365041 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-015c9704-39e3-497a-8280-f49533ca3610 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.169765 Sep 08 14:04:05.365636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 836/1780] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:05 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:05.429375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-97ed7337-3aca-40c2-8793-d2d52771dab2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:05.431560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-97ed7337-3aca-40c2-8793-d2d52771dab2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:05.438235 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-97ed7337-3aca-40c2-8793-d2d52771dab2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:05.438692 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-97ed7337-3aca-40c2-8793-d2d52771dab2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:05.439105 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-97ed7337-3aca-40c2-8793-d2d52771dab2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:05.439807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-97ed7337-3aca-40c2-8793-d2d52771dab2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:05.440258 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-97ed7337-3aca-40c2-8793-d2d52771dab2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:05.440685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-97ed7337-3aca-40c2-8793-d2d52771dab2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:05.484436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-97ed7337-3aca-40c2-8793-d2d52771dab2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.060637 Sep 08 14:04:05.485021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 945/1781] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:05 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:05.724964 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-be720d05-bdac-4ec4-b3a9-9970170de746 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:05.725590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-be720d05-bdac-4ec4-b3a9-9970170de746 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:05.732574 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be720d05-bdac-4ec4-b3a9-9970170de746 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:05.733059 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be720d05-bdac-4ec4-b3a9-9970170de746 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:05.733490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be720d05-bdac-4ec4-b3a9-9970170de746 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:05.734215 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be720d05-bdac-4ec4-b3a9-9970170de746 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:05.734667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be720d05-bdac-4ec4-b3a9-9970170de746 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:05.735087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be720d05-bdac-4ec4-b3a9-9970170de746 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:05.798058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-be720d05-bdac-4ec4-b3a9-9970170de746 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:05.806197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-be720d05-bdac-4ec4-b3a9-9970170de746 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.084975 Sep 08 14:04:05.806808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 837/1782] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:05 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:06.086541 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e6460507-3200-4622-96bf-7b5843d8efde tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:06.088849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e6460507-3200-4622-96bf-7b5843d8efde tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:06.095663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6460507-3200-4622-96bf-7b5843d8efde tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:06.096146 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6460507-3200-4622-96bf-7b5843d8efde tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:06.096582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6460507-3200-4622-96bf-7b5843d8efde tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:06.097283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6460507-3200-4622-96bf-7b5843d8efde tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:06.097731 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6460507-3200-4622-96bf-7b5843d8efde tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:06.098170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6460507-3200-4622-96bf-7b5843d8efde tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:06.129268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e6460507-3200-4622-96bf-7b5843d8efde tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046697 Sep 08 14:04:06.129955 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 946/1783] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:06 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:06.166831 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-dc8d0bbd-58e3-42f6-a898-53ca0b5f1aac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:06.169108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-dc8d0bbd-58e3-42f6-a898-53ca0b5f1aac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:06.175720 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dc8d0bbd-58e3-42f6-a898-53ca0b5f1aac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:06.176200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dc8d0bbd-58e3-42f6-a898-53ca0b5f1aac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:06.176628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dc8d0bbd-58e3-42f6-a898-53ca0b5f1aac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:06.177315 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dc8d0bbd-58e3-42f6-a898-53ca0b5f1aac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:06.177763 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dc8d0bbd-58e3-42f6-a898-53ca0b5f1aac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:06.178214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dc8d0bbd-58e3-42f6-a898-53ca0b5f1aac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:06.204826 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:06.209418 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-dc8d0bbd-58e3-42f6-a898-53ca0b5f1aac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046711 Sep 08 14:04:06.209967 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 838/1784] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:06 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:06.389580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-73e530a7-1a35-4f1b-8cec-b7189d38673c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:06.390211 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-73e530a7-1a35-4f1b-8cec-b7189d38673c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:06.396246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73e530a7-1a35-4f1b-8cec-b7189d38673c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:06.396732 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73e530a7-1a35-4f1b-8cec-b7189d38673c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:06.397168 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73e530a7-1a35-4f1b-8cec-b7189d38673c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:06.397869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73e530a7-1a35-4f1b-8cec-b7189d38673c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:06.398366 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73e530a7-1a35-4f1b-8cec-b7189d38673c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:06.398793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73e530a7-1a35-4f1b-8cec-b7189d38673c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:06.498671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5c25a253-3b80-4bb9-9ca4-5801ee6897a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:06.500855 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5c25a253-3b80-4bb9-9ca4-5801ee6897a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:06.508220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5c25a253-3b80-4bb9-9ca4-5801ee6897a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:06.508845 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5c25a253-3b80-4bb9-9ca4-5801ee6897a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:06.509535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5c25a253-3b80-4bb9-9ca4-5801ee6897a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:06.510286 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5c25a253-3b80-4bb9-9ca4-5801ee6897a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:06.510889 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5c25a253-3b80-4bb9-9ca4-5801ee6897a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:06.511487 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5c25a253-3b80-4bb9-9ca4-5801ee6897a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:06.545563 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5c25a253-3b80-4bb9-9ca4-5801ee6897a5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052394 Sep 08 14:04:06.546377 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 839/1785] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:06 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:04:06.552944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-73e530a7-1a35-4f1b-8cec-b7189d38673c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:06.561857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-73e530a7-1a35-4f1b-8cec-b7189d38673c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.176776 Sep 08 14:04:06.562555 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 947/1786] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:06 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:06.829978 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8269bfdd-5d5a-45e5-99fc-2bd7e232b8bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:06.830514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8269bfdd-5d5a-45e5-99fc-2bd7e232b8bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:06.836586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8269bfdd-5d5a-45e5-99fc-2bd7e232b8bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:06.837144 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8269bfdd-5d5a-45e5-99fc-2bd7e232b8bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:06.837580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8269bfdd-5d5a-45e5-99fc-2bd7e232b8bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:06.838319 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8269bfdd-5d5a-45e5-99fc-2bd7e232b8bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:06.838769 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8269bfdd-5d5a-45e5-99fc-2bd7e232b8bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:06.839288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8269bfdd-5d5a-45e5-99fc-2bd7e232b8bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:06.908323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8269bfdd-5d5a-45e5-99fc-2bd7e232b8bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:06.916585 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8269bfdd-5d5a-45e5-99fc-2bd7e232b8bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.090557 Sep 08 14:04:06.917174 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 840/1787] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:06 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:07.143704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a7babba3-62e9-488c-a9ad-96b259cd443d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:07.146939 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a7babba3-62e9-488c-a9ad-96b259cd443d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:07.154480 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7babba3-62e9-488c-a9ad-96b259cd443d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:07.154989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7babba3-62e9-488c-a9ad-96b259cd443d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:07.155427 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7babba3-62e9-488c-a9ad-96b259cd443d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:07.156143 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7babba3-62e9-488c-a9ad-96b259cd443d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:07.156599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7babba3-62e9-488c-a9ad-96b259cd443d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:07.157026 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7babba3-62e9-488c-a9ad-96b259cd443d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:07.188616 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a7babba3-62e9-488c-a9ad-96b259cd443d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050929 Sep 08 14:04:07.189186 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 948/1788] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:07 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:07.222932 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-39f2d89c-3803-4624-97b6-c4a2ed8770e1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:07.225095 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-39f2d89c-3803-4624-97b6-c4a2ed8770e1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:07.231758 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39f2d89c-3803-4624-97b6-c4a2ed8770e1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:07.232847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39f2d89c-3803-4624-97b6-c4a2ed8770e1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:07.233283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39f2d89c-3803-4624-97b6-c4a2ed8770e1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:07.233981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39f2d89c-3803-4624-97b6-c4a2ed8770e1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:07.234455 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39f2d89c-3803-4624-97b6-c4a2ed8770e1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:07.234875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39f2d89c-3803-4624-97b6-c4a2ed8770e1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:07.271033 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-39f2d89c-3803-4624-97b6-c4a2ed8770e1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052004 Sep 08 14:04:07.271619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 841/1789] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:07 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:07.558722 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-32680bb1-5383-483b-ab05-a9190441feb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:07.560906 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-32680bb1-5383-483b-ab05-a9190441feb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:07.567080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32680bb1-5383-483b-ab05-a9190441feb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:07.567570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32680bb1-5383-483b-ab05-a9190441feb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:07.568010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32680bb1-5383-483b-ab05-a9190441feb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:07.568715 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32680bb1-5383-483b-ab05-a9190441feb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:07.569165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32680bb1-5383-483b-ab05-a9190441feb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:07.569589 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32680bb1-5383-483b-ab05-a9190441feb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:07.587787 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-75daaf95-10fd-4a96-91d0-d1280b19638e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:07.588405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-75daaf95-10fd-4a96-91d0-d1280b19638e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:07.598120 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-75daaf95-10fd-4a96-91d0-d1280b19638e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:07.598726 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-75daaf95-10fd-4a96-91d0-d1280b19638e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:07.599261 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-75daaf95-10fd-4a96-91d0-d1280b19638e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:07.599989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-75daaf95-10fd-4a96-91d0-d1280b19638e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:07.600448 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-75daaf95-10fd-4a96-91d0-d1280b19638e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:07.600909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-75daaf95-10fd-4a96-91d0-d1280b19638e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:07.604843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-32680bb1-5383-483b-ab05-a9190441feb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050334 Sep 08 14:04:07.607668 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 949/1790] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:07 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:07.756766 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-75daaf95-10fd-4a96-91d0-d1280b19638e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:07.768253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-75daaf95-10fd-4a96-91d0-d1280b19638e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.184847 Sep 08 14:04:07.768846 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 842/1791] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:07 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:07.941093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6f3b337c-ac98-4df8-84e9-b12cdf5074e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:07.941739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6f3b337c-ac98-4df8-84e9-b12cdf5074e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:07.948231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f3b337c-ac98-4df8-84e9-b12cdf5074e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:07.948752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f3b337c-ac98-4df8-84e9-b12cdf5074e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:07.949220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f3b337c-ac98-4df8-84e9-b12cdf5074e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:07.949974 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f3b337c-ac98-4df8-84e9-b12cdf5074e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:07.950518 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f3b337c-ac98-4df8-84e9-b12cdf5074e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:07.951760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f3b337c-ac98-4df8-84e9-b12cdf5074e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:08.017720 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6f3b337c-ac98-4df8-84e9-b12cdf5074e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:08.026591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6f3b337c-ac98-4df8-84e9-b12cdf5074e4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.089415 Sep 08 14:04:08.027275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 950/1792] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:07 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:08.201518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5e336d80-7260-4057-bf05-4c8ace3a43f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:08.203746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5e336d80-7260-4057-bf05-4c8ace3a43f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:08.210040 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e336d80-7260-4057-bf05-4c8ace3a43f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:08.210556 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e336d80-7260-4057-bf05-4c8ace3a43f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:08.210992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e336d80-7260-4057-bf05-4c8ace3a43f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:08.211773 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e336d80-7260-4057-bf05-4c8ace3a43f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:08.212259 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e336d80-7260-4057-bf05-4c8ace3a43f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:08.212952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e336d80-7260-4057-bf05-4c8ace3a43f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:08.245561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5e336d80-7260-4057-bf05-4c8ace3a43f8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047976 Sep 08 14:04:08.246261 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 843/1793] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:08 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:08.283870 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c147825c-b417-46fe-9f9d-3e02ddc1a7dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:08.285948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c147825c-b417-46fe-9f9d-3e02ddc1a7dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:08.292698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c147825c-b417-46fe-9f9d-3e02ddc1a7dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:08.293176 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c147825c-b417-46fe-9f9d-3e02ddc1a7dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:08.293606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c147825c-b417-46fe-9f9d-3e02ddc1a7dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:08.294347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c147825c-b417-46fe-9f9d-3e02ddc1a7dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:08.295319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c147825c-b417-46fe-9f9d-3e02ddc1a7dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:08.295746 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c147825c-b417-46fe-9f9d-3e02ddc1a7dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:08.327660 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c147825c-b417-46fe-9f9d-3e02ddc1a7dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047255 Sep 08 14:04:08.328241 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 951/1794] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:08 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:08.617251 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9062cc8b-3770-42b6-972a-571548275c1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:08.619704 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9062cc8b-3770-42b6-972a-571548275c1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:08.626013 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9062cc8b-3770-42b6-972a-571548275c1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:08.626539 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9062cc8b-3770-42b6-972a-571548275c1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:08.627011 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9062cc8b-3770-42b6-972a-571548275c1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:08.627731 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9062cc8b-3770-42b6-972a-571548275c1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:08.628178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9062cc8b-3770-42b6-972a-571548275c1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:08.628601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9062cc8b-3770-42b6-972a-571548275c1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:08.661554 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9062cc8b-3770-42b6-972a-571548275c1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048324 Sep 08 14:04:08.662130 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 844/1795] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:08 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:08.796368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fad02897-f4d9-4d15-87a8-397354fa388b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:08.797452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fad02897-f4d9-4d15-87a8-397354fa388b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:08.805162 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fad02897-f4d9-4d15-87a8-397354fa388b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:08.805645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fad02897-f4d9-4d15-87a8-397354fa388b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:08.806081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fad02897-f4d9-4d15-87a8-397354fa388b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:08.806808 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fad02897-f4d9-4d15-87a8-397354fa388b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:08.807256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fad02897-f4d9-4d15-87a8-397354fa388b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:08.807673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fad02897-f4d9-4d15-87a8-397354fa388b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:08.949326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fad02897-f4d9-4d15-87a8-397354fa388b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:08.958105 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fad02897-f4d9-4d15-87a8-397354fa388b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.169597 Sep 08 14:04:08.958717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 952/1796] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:08 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:09.050661 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-94636f92-b05b-4679-bf0b-d02873305690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:09.051651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-94636f92-b05b-4679-bf0b-d02873305690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:09.057899 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94636f92-b05b-4679-bf0b-d02873305690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:09.058416 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94636f92-b05b-4679-bf0b-d02873305690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:09.059054 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94636f92-b05b-4679-bf0b-d02873305690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:09.059759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94636f92-b05b-4679-bf0b-d02873305690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:09.060235 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94636f92-b05b-4679-bf0b-d02873305690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:09.060897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94636f92-b05b-4679-bf0b-d02873305690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:09.133702 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-94636f92-b05b-4679-bf0b-d02873305690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:09.144891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-94636f92-b05b-4679-bf0b-d02873305690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.096610 Sep 08 14:04:09.144891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 845/1797] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:09 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:09.259184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-249773d8-7f2b-4f2a-bcd7-4620711c48db tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:09.262376 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-249773d8-7f2b-4f2a-bcd7-4620711c48db tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:09.273425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-249773d8-7f2b-4f2a-bcd7-4620711c48db tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:09.273921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-249773d8-7f2b-4f2a-bcd7-4620711c48db tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:09.274392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-249773d8-7f2b-4f2a-bcd7-4620711c48db tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:09.275109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-249773d8-7f2b-4f2a-bcd7-4620711c48db tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:09.275570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-249773d8-7f2b-4f2a-bcd7-4620711c48db tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:09.275996 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-249773d8-7f2b-4f2a-bcd7-4620711c48db tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:09.308179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-249773d8-7f2b-4f2a-bcd7-4620711c48db tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053781 Sep 08 14:04:09.308878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 953/1798] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:09 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:09.343475 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4e494ff6-b098-442e-bb27-401330fc879e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:09.345696 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4e494ff6-b098-442e-bb27-401330fc879e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:09.352703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4e494ff6-b098-442e-bb27-401330fc879e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:09.353194 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4e494ff6-b098-442e-bb27-401330fc879e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:09.353680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4e494ff6-b098-442e-bb27-401330fc879e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:09.354544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4e494ff6-b098-442e-bb27-401330fc879e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:09.355289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4e494ff6-b098-442e-bb27-401330fc879e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:09.355986 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4e494ff6-b098-442e-bb27-401330fc879e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:09.389796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4e494ff6-b098-442e-bb27-401330fc879e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051182 Sep 08 14:04:09.390442 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 846/1799] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:09 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:09.675196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e39ca2a2-64b0-445d-a38e-58c3148cec73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:09.677661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e39ca2a2-64b0-445d-a38e-58c3148cec73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:09.684140 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e39ca2a2-64b0-445d-a38e-58c3148cec73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:09.684688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e39ca2a2-64b0-445d-a38e-58c3148cec73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:09.685777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e39ca2a2-64b0-445d-a38e-58c3148cec73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:09.686563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e39ca2a2-64b0-445d-a38e-58c3148cec73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:09.687089 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e39ca2a2-64b0-445d-a38e-58c3148cec73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:09.687556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e39ca2a2-64b0-445d-a38e-58c3148cec73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:09.730514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e39ca2a2-64b0-445d-a38e-58c3148cec73 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058962 Sep 08 14:04:09.732776 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 954/1800] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:09 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:09.983864 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0553795f-7d60-4522-b180-b56c9929d822 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:09.984543 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0553795f-7d60-4522-b180-b56c9929d822 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:09.991415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0553795f-7d60-4522-b180-b56c9929d822 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:09.991901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0553795f-7d60-4522-b180-b56c9929d822 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:09.992337 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0553795f-7d60-4522-b180-b56c9929d822 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:09.993136 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0553795f-7d60-4522-b180-b56c9929d822 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:09.993636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0553795f-7d60-4522-b180-b56c9929d822 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:09.994107 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0553795f-7d60-4522-b180-b56c9929d822 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:10.158182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0553795f-7d60-4522-b180-b56c9929d822 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:10.168082 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0553795f-7d60-4522-b180-b56c9929d822 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.188890 Sep 08 14:04:10.168355 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-751ea468-7603-4be7-b572-7ea63c908567 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:10.178236 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 847/1801] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:09 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:10.178506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-751ea468-7603-4be7-b572-7ea63c908567 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:10.178506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-751ea468-7603-4be7-b572-7ea63c908567 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:10.178506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-751ea468-7603-4be7-b572-7ea63c908567 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:10.178869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-751ea468-7603-4be7-b572-7ea63c908567 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:10.179475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-751ea468-7603-4be7-b572-7ea63c908567 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:10.179990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-751ea468-7603-4be7-b572-7ea63c908567 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:10.180572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-751ea468-7603-4be7-b572-7ea63c908567 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:10.253290 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-751ea468-7603-4be7-b572-7ea63c908567 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:10.261560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-751ea468-7603-4be7-b572-7ea63c908567 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.097657 Sep 08 14:04:10.262169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 955/1802] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:10 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:10.321755 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4a57b2f9-873c-46cc-bf1d-99a7a0007c25 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:10.324099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4a57b2f9-873c-46cc-bf1d-99a7a0007c25 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:10.331082 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a57b2f9-873c-46cc-bf1d-99a7a0007c25 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:10.331579 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a57b2f9-873c-46cc-bf1d-99a7a0007c25 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:10.332021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a57b2f9-873c-46cc-bf1d-99a7a0007c25 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:10.332737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a57b2f9-873c-46cc-bf1d-99a7a0007c25 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:10.333192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a57b2f9-873c-46cc-bf1d-99a7a0007c25 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:10.333766 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a57b2f9-873c-46cc-bf1d-99a7a0007c25 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:10.372915 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4a57b2f9-873c-46cc-bf1d-99a7a0007c25 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055120 Sep 08 14:04:10.373589 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 848/1803] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:10 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:10.408445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-359ff92e-be51-4c94-b934-6a0190f83010 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:10.410757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-359ff92e-be51-4c94-b934-6a0190f83010 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:10.417655 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-359ff92e-be51-4c94-b934-6a0190f83010 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:10.418182 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-359ff92e-be51-4c94-b934-6a0190f83010 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:10.418637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-359ff92e-be51-4c94-b934-6a0190f83010 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:10.419379 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-359ff92e-be51-4c94-b934-6a0190f83010 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:10.419837 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-359ff92e-be51-4c94-b934-6a0190f83010 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:10.420273 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-359ff92e-be51-4c94-b934-6a0190f83010 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:10.457556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-359ff92e-be51-4c94-b934-6a0190f83010 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053293 Sep 08 14:04:10.465030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 956/1804] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:10 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:10.745852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8d3fe987-b8ac-401f-9248-534d1c200a84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:10.749355 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8d3fe987-b8ac-401f-9248-534d1c200a84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:10.758245 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d3fe987-b8ac-401f-9248-534d1c200a84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:10.758747 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d3fe987-b8ac-401f-9248-534d1c200a84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:10.759161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d3fe987-b8ac-401f-9248-534d1c200a84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:10.759874 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d3fe987-b8ac-401f-9248-534d1c200a84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:10.760353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d3fe987-b8ac-401f-9248-534d1c200a84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:10.760918 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d3fe987-b8ac-401f-9248-534d1c200a84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:10.795250 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8d3fe987-b8ac-401f-9248-534d1c200a84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054136 Sep 08 14:04:10.795999 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 849/1805] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:10 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:11.196933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1da4006d-d948-49e6-92ce-df2580d54aef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:11.197647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1da4006d-d948-49e6-92ce-df2580d54aef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:11.204198 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1da4006d-d948-49e6-92ce-df2580d54aef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:11.204682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1da4006d-d948-49e6-92ce-df2580d54aef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:11.205117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1da4006d-d948-49e6-92ce-df2580d54aef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:11.205818 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1da4006d-d948-49e6-92ce-df2580d54aef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:11.206314 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1da4006d-d948-49e6-92ce-df2580d54aef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:11.206743 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1da4006d-d948-49e6-92ce-df2580d54aef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:11.284201 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-98a9585b-51a9-4736-ad9c-a40dcca84a60 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:11.284917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-98a9585b-51a9-4736-ad9c-a40dcca84a60 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:11.291296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-98a9585b-51a9-4736-ad9c-a40dcca84a60 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:11.292072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-98a9585b-51a9-4736-ad9c-a40dcca84a60 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:11.292257 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-98a9585b-51a9-4736-ad9c-a40dcca84a60 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:11.292919 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-98a9585b-51a9-4736-ad9c-a40dcca84a60 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:11.293370 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-98a9585b-51a9-4736-ad9c-a40dcca84a60 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:11.293787 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-98a9585b-51a9-4736-ad9c-a40dcca84a60 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:11.370216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1da4006d-d948-49e6-92ce-df2580d54aef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:11.381690 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-98a9585b-51a9-4736-ad9c-a40dcca84a60 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:11.382849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1da4006d-d948-49e6-92ce-df2580d54aef tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.189910 Sep 08 14:04:11.383765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 957/1806] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:11 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:11.389691 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fcf644e2-3cc1-466a-b617-8ab8e17d602b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:11.392049 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fcf644e2-3cc1-466a-b617-8ab8e17d602b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:11.392802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-98a9585b-51a9-4736-ad9c-a40dcca84a60 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.112544 Sep 08 14:04:11.393545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 850/1807] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:11 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:11.403451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fcf644e2-3cc1-466a-b617-8ab8e17d602b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:11.403947 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fcf644e2-3cc1-466a-b617-8ab8e17d602b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:11.404497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fcf644e2-3cc1-466a-b617-8ab8e17d602b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:11.405240 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fcf644e2-3cc1-466a-b617-8ab8e17d602b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:11.406060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fcf644e2-3cc1-466a-b617-8ab8e17d602b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:11.406313 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fcf644e2-3cc1-466a-b617-8ab8e17d602b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:11.440498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fcf644e2-3cc1-466a-b617-8ab8e17d602b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055932 Sep 08 14:04:11.441246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 958/1808] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:11 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:11.474896 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2da6ee90-4292-4a2d-90e5-9658bf5a65a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:11.477028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2da6ee90-4292-4a2d-90e5-9658bf5a65a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:11.483877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2da6ee90-4292-4a2d-90e5-9658bf5a65a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:11.484426 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2da6ee90-4292-4a2d-90e5-9658bf5a65a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:11.484986 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2da6ee90-4292-4a2d-90e5-9658bf5a65a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:11.485717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2da6ee90-4292-4a2d-90e5-9658bf5a65a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:11.486191 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2da6ee90-4292-4a2d-90e5-9658bf5a65a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:11.486626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2da6ee90-4292-4a2d-90e5-9658bf5a65a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:11.519486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2da6ee90-4292-4a2d-90e5-9658bf5a65a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049021 Sep 08 14:04:11.520128 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 851/1809] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:11 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:11.891876 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-afa3e604-4a38-468c-8185-16c1ae60f4b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:11.894160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-afa3e604-4a38-468c-8185-16c1ae60f4b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:11.900529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-afa3e604-4a38-468c-8185-16c1ae60f4b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:11.901058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-afa3e604-4a38-468c-8185-16c1ae60f4b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:11.901507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-afa3e604-4a38-468c-8185-16c1ae60f4b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:11.902271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-afa3e604-4a38-468c-8185-16c1ae60f4b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:11.902740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-afa3e604-4a38-468c-8185-16c1ae60f4b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:11.903173 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-afa3e604-4a38-468c-8185-16c1ae60f4b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:11.919886 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:11.937351 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-afa3e604-4a38-468c-8185-16c1ae60f4b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.131865 Sep 08 14:04:11.937935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 959/1810] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:11 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:12.409553 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4899d9cf-083a-4b14-8ffe-b3458d4de9bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:12.410460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4899d9cf-083a-4b14-8ffe-b3458d4de9bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:12.417083 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ab93bdb0-5a24-4e20-9bd3-ee461025b218 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:12.417459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4899d9cf-083a-4b14-8ffe-b3458d4de9bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:12.417695 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ab93bdb0-5a24-4e20-9bd3-ee461025b218 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:12.417954 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4899d9cf-083a-4b14-8ffe-b3458d4de9bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:12.418437 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4899d9cf-083a-4b14-8ffe-b3458d4de9bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:12.419154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4899d9cf-083a-4b14-8ffe-b3458d4de9bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:12.419621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4899d9cf-083a-4b14-8ffe-b3458d4de9bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:12.420050 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4899d9cf-083a-4b14-8ffe-b3458d4de9bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:12.425407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab93bdb0-5a24-4e20-9bd3-ee461025b218 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:12.426092 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab93bdb0-5a24-4e20-9bd3-ee461025b218 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:12.426638 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab93bdb0-5a24-4e20-9bd3-ee461025b218 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:12.427396 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab93bdb0-5a24-4e20-9bd3-ee461025b218 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:12.427912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab93bdb0-5a24-4e20-9bd3-ee461025b218 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:12.428371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab93bdb0-5a24-4e20-9bd3-ee461025b218 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:12.503115 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ab93bdb0-5a24-4e20-9bd3-ee461025b218 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:12.511986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ab93bdb0-5a24-4e20-9bd3-ee461025b218 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.098985 Sep 08 14:04:12.512500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 960/1811] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:12 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:12.518008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-06f837c9-1c28-48d5-ba04-18a0d84f4a3b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:12.520238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-06f837c9-1c28-48d5-ba04-18a0d84f4a3b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:12.527193 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06f837c9-1c28-48d5-ba04-18a0d84f4a3b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:12.527193 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06f837c9-1c28-48d5-ba04-18a0d84f4a3b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:12.527822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06f837c9-1c28-48d5-ba04-18a0d84f4a3b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:12.528834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06f837c9-1c28-48d5-ba04-18a0d84f4a3b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:12.529317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06f837c9-1c28-48d5-ba04-18a0d84f4a3b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:12.529759 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06f837c9-1c28-48d5-ba04-18a0d84f4a3b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:12.563459 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-06f837c9-1c28-48d5-ba04-18a0d84f4a3b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049214 Sep 08 14:04:12.564068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 961/1812] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:12 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:12.569064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c7734fee-a2af-4677-bd56-4b4b7c7e7f5e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:12.571222 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c7734fee-a2af-4677-bd56-4b4b7c7e7f5e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:12.578276 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c7734fee-a2af-4677-bd56-4b4b7c7e7f5e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:12.584106 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c7734fee-a2af-4677-bd56-4b4b7c7e7f5e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:12.584313 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4899d9cf-083a-4b14-8ffe-b3458d4de9bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:12.584455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c7734fee-a2af-4677-bd56-4b4b7c7e7f5e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:12.584455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c7734fee-a2af-4677-bd56-4b4b7c7e7f5e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:12.584455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c7734fee-a2af-4677-bd56-4b4b7c7e7f5e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:12.584455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c7734fee-a2af-4677-bd56-4b4b7c7e7f5e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:12.587690 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4899d9cf-083a-4b14-8ffe-b3458d4de9bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.183863 Sep 08 14:04:12.588286 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 852/1813] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:12 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:12.613822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c7734fee-a2af-4677-bd56-4b4b7c7e7f5e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048667 Sep 08 14:04:12.614422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 962/1814] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:12 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:12.950748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-35bb7e2b-aa0b-446c-bf1f-6417407dcb92 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:12.953424 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-35bb7e2b-aa0b-446c-bf1f-6417407dcb92 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:12.960134 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-35bb7e2b-aa0b-446c-bf1f-6417407dcb92 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:12.960650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-35bb7e2b-aa0b-446c-bf1f-6417407dcb92 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:12.961089 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-35bb7e2b-aa0b-446c-bf1f-6417407dcb92 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:12.961804 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-35bb7e2b-aa0b-446c-bf1f-6417407dcb92 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:12.962340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-35bb7e2b-aa0b-446c-bf1f-6417407dcb92 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:12.962920 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-35bb7e2b-aa0b-446c-bf1f-6417407dcb92 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:13.003646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-35bb7e2b-aa0b-446c-bf1f-6417407dcb92 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057258 Sep 08 14:04:13.004419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 853/1815] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:12 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:13.536194 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5ea0db43-0b68-46a7-b56e-de18e5ba734e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:13.536815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5ea0db43-0b68-46a7-b56e-de18e5ba734e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:13.547162 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5ea0db43-0b68-46a7-b56e-de18e5ba734e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:13.547162 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5ea0db43-0b68-46a7-b56e-de18e5ba734e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:13.547162 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5ea0db43-0b68-46a7-b56e-de18e5ba734e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:13.547162 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5ea0db43-0b68-46a7-b56e-de18e5ba734e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:13.550478 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5ea0db43-0b68-46a7-b56e-de18e5ba734e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:13.550478 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5ea0db43-0b68-46a7-b56e-de18e5ba734e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:13.578941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-53c20f5f-770e-43c6-b1f7-2541bdf15c30 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:13.581175 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-53c20f5f-770e-43c6-b1f7-2541bdf15c30 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:13.589594 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-53c20f5f-770e-43c6-b1f7-2541bdf15c30 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:13.590333 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-53c20f5f-770e-43c6-b1f7-2541bdf15c30 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:13.591444 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-53c20f5f-770e-43c6-b1f7-2541bdf15c30 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:13.592209 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-53c20f5f-770e-43c6-b1f7-2541bdf15c30 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:13.592695 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-53c20f5f-770e-43c6-b1f7-2541bdf15c30 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:13.593172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-53c20f5f-770e-43c6-b1f7-2541bdf15c30 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:13.620438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5ea0db43-0b68-46a7-b56e-de18e5ba734e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:13.628893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5ea0db43-0b68-46a7-b56e-de18e5ba734e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.096795 Sep 08 14:04:13.629475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 963/1816] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:13 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:13.638456 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c362b7fc-27ec-4315-ad34-9ac1025d7192 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:13.638456 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c362b7fc-27ec-4315-ad34-9ac1025d7192 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:13.641977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c362b7fc-27ec-4315-ad34-9ac1025d7192 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:13.642499 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c362b7fc-27ec-4315-ad34-9ac1025d7192 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:13.642934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c362b7fc-27ec-4315-ad34-9ac1025d7192 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:13.643626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c362b7fc-27ec-4315-ad34-9ac1025d7192 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:13.644634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c362b7fc-27ec-4315-ad34-9ac1025d7192 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:13.644634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c362b7fc-27ec-4315-ad34-9ac1025d7192 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:13.649626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-53c20f5f-770e-43c6-b1f7-2541bdf15c30 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.076981 Sep 08 14:04:13.650341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 854/1817] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:13 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:13.657368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2c24b250-a4f0-4156-aea2-65e14e551e08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:13.660337 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2c24b250-a4f0-4156-aea2-65e14e551e08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:13.667157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2c24b250-a4f0-4156-aea2-65e14e551e08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:13.667688 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2c24b250-a4f0-4156-aea2-65e14e551e08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:13.668350 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2c24b250-a4f0-4156-aea2-65e14e551e08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:13.669299 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2c24b250-a4f0-4156-aea2-65e14e551e08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:13.669764 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2c24b250-a4f0-4156-aea2-65e14e551e08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:13.670209 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2c24b250-a4f0-4156-aea2-65e14e551e08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:13.691866 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:13.709168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2c24b250-a4f0-4156-aea2-65e14e551e08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057410 Sep 08 14:04:13.709953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 855/1818] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:13 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:13.813348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c362b7fc-27ec-4315-ad34-9ac1025d7192 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:13.823347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c362b7fc-27ec-4315-ad34-9ac1025d7192 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.192775 Sep 08 14:04:13.823964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 964/1819] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:13 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:14.018858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d8f37ff2-35ec-495f-81cd-09691a2ebee3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:14.021357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d8f37ff2-35ec-495f-81cd-09691a2ebee3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:14.028890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d8f37ff2-35ec-495f-81cd-09691a2ebee3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:14.029404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d8f37ff2-35ec-495f-81cd-09691a2ebee3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:14.029834 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d8f37ff2-35ec-495f-81cd-09691a2ebee3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:14.030567 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d8f37ff2-35ec-495f-81cd-09691a2ebee3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:14.031015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d8f37ff2-35ec-495f-81cd-09691a2ebee3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:14.031438 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d8f37ff2-35ec-495f-81cd-09691a2ebee3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:14.063620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d8f37ff2-35ec-495f-81cd-09691a2ebee3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048913 Sep 08 14:04:14.065157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 856/1820] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:14 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:14.654444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-31daad52-f072-4379-b92a-154b10bee26d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:14.655059 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-31daad52-f072-4379-b92a-154b10bee26d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:14.663627 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31daad52-f072-4379-b92a-154b10bee26d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:14.664162 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31daad52-f072-4379-b92a-154b10bee26d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:14.664601 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31daad52-f072-4379-b92a-154b10bee26d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:14.665414 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31daad52-f072-4379-b92a-154b10bee26d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:14.665580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-db016d92-e4d3-462e-877a-52bce2a9603c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:14.666073 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31daad52-f072-4379-b92a-154b10bee26d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:14.666392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31daad52-f072-4379-b92a-154b10bee26d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:14.667675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-db016d92-e4d3-462e-877a-52bce2a9603c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:14.674689 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-db016d92-e4d3-462e-877a-52bce2a9603c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:14.675167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-db016d92-e4d3-462e-877a-52bce2a9603c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:14.675597 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-db016d92-e4d3-462e-877a-52bce2a9603c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:14.676290 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-db016d92-e4d3-462e-877a-52bce2a9603c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:14.676735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-db016d92-e4d3-462e-877a-52bce2a9603c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:14.677149 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-db016d92-e4d3-462e-877a-52bce2a9603c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:14.711928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-db016d92-e4d3-462e-877a-52bce2a9603c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053084 Sep 08 14:04:14.712508 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 857/1821] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:14 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:14.726371 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1370f8d0-d67b-41ce-960a-e7bdcc773216 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:14.730360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1370f8d0-d67b-41ce-960a-e7bdcc773216 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:14.740223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-31daad52-f072-4379-b92a-154b10bee26d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:14.741289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1370f8d0-d67b-41ce-960a-e7bdcc773216 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:14.742157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1370f8d0-d67b-41ce-960a-e7bdcc773216 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:14.742937 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1370f8d0-d67b-41ce-960a-e7bdcc773216 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:14.744429 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1370f8d0-d67b-41ce-960a-e7bdcc773216 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:14.745232 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1370f8d0-d67b-41ce-960a-e7bdcc773216 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:14.746023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1370f8d0-d67b-41ce-960a-e7bdcc773216 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:14.757030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-31daad52-f072-4379-b92a-154b10bee26d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.106559 Sep 08 14:04:14.757626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 965/1822] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:14 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:14.784451 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1370f8d0-d67b-41ce-960a-e7bdcc773216 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064718 Sep 08 14:04:14.784451 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 858/1823] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:14 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:14.850201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0657cf79-f88c-4ea6-a14f-732926c84361 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:14.850811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0657cf79-f88c-4ea6-a14f-732926c84361 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:14.859807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0657cf79-f88c-4ea6-a14f-732926c84361 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:14.860305 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0657cf79-f88c-4ea6-a14f-732926c84361 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:14.860744 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0657cf79-f88c-4ea6-a14f-732926c84361 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:14.861444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0657cf79-f88c-4ea6-a14f-732926c84361 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:14.861901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0657cf79-f88c-4ea6-a14f-732926c84361 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:14.862373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0657cf79-f88c-4ea6-a14f-732926c84361 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:15.004653 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0657cf79-f88c-4ea6-a14f-732926c84361 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:15.015011 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0657cf79-f88c-4ea6-a14f-732926c84361 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.168709 Sep 08 14:04:15.015590 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 966/1824] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:14 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:15.078662 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3e17c53f-6fa0-491b-a0d0-ce45e64eaa63 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:15.080816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3e17c53f-6fa0-491b-a0d0-ce45e64eaa63 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:15.087260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e17c53f-6fa0-491b-a0d0-ce45e64eaa63 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:15.087818 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e17c53f-6fa0-491b-a0d0-ce45e64eaa63 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:15.088260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e17c53f-6fa0-491b-a0d0-ce45e64eaa63 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:15.088974 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e17c53f-6fa0-491b-a0d0-ce45e64eaa63 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:15.089427 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e17c53f-6fa0-491b-a0d0-ce45e64eaa63 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:15.089857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e17c53f-6fa0-491b-a0d0-ce45e64eaa63 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:15.125081 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3e17c53f-6fa0-491b-a0d0-ce45e64eaa63 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051485 Sep 08 14:04:15.125678 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 859/1825] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:15 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:15.727013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-120284df-fb89-4dc2-9199-22dd191b89c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:15.729234 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-120284df-fb89-4dc2-9199-22dd191b89c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:15.738006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-120284df-fb89-4dc2-9199-22dd191b89c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:15.738938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-120284df-fb89-4dc2-9199-22dd191b89c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:15.739758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-120284df-fb89-4dc2-9199-22dd191b89c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:15.740940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-120284df-fb89-4dc2-9199-22dd191b89c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:15.741420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-120284df-fb89-4dc2-9199-22dd191b89c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:15.741865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-120284df-fb89-4dc2-9199-22dd191b89c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:15.777681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-120284df-fb89-4dc2-9199-22dd191b89c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056563 Sep 08 14:04:15.778869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 967/1826] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:15 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:15.786696 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0646c9dc-26bf-4c22-8420-fcf31484e62e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:15.787363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0646c9dc-26bf-4c22-8420-fcf31484e62e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:15.795928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0646c9dc-26bf-4c22-8420-fcf31484e62e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:15.796488 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0b34dbe5-2979-4664-a943-3d80ab8ce3c9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:15.796706 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0646c9dc-26bf-4c22-8420-fcf31484e62e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:15.797104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0646c9dc-26bf-4c22-8420-fcf31484e62e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:15.798268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0646c9dc-26bf-4c22-8420-fcf31484e62e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:15.798818 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0646c9dc-26bf-4c22-8420-fcf31484e62e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:15.798981 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0b34dbe5-2979-4664-a943-3d80ab8ce3c9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:15.799383 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0646c9dc-26bf-4c22-8420-fcf31484e62e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:15.806527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b34dbe5-2979-4664-a943-3d80ab8ce3c9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:15.806527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b34dbe5-2979-4664-a943-3d80ab8ce3c9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:15.807642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b34dbe5-2979-4664-a943-3d80ab8ce3c9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:15.808334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b34dbe5-2979-4664-a943-3d80ab8ce3c9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:15.808792 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b34dbe5-2979-4664-a943-3d80ab8ce3c9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:15.809219 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0b34dbe5-2979-4664-a943-3d80ab8ce3c9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:15.846564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0b34dbe5-2979-4664-a943-3d80ab8ce3c9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053913 Sep 08 14:04:15.847157 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 968/1827] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:15 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:15.883891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0646c9dc-26bf-4c22-8420-fcf31484e62e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:15.892887 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0646c9dc-26bf-4c22-8420-fcf31484e62e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.110113 Sep 08 14:04:15.893386 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 860/1828] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:15 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:16.039751 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f0c78281-0894-4f4a-85e9-9cb25eca0bba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:16.040352 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f0c78281-0894-4f4a-85e9-9cb25eca0bba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:16.046430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0c78281-0894-4f4a-85e9-9cb25eca0bba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:16.047212 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0c78281-0894-4f4a-85e9-9cb25eca0bba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:16.047649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0c78281-0894-4f4a-85e9-9cb25eca0bba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:16.048343 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0c78281-0894-4f4a-85e9-9cb25eca0bba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:16.048790 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0c78281-0894-4f4a-85e9-9cb25eca0bba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:16.049208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0c78281-0894-4f4a-85e9-9cb25eca0bba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:16.145560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c6bffa24-38c7-4244-9893-873902d61202 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:16.149138 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c6bffa24-38c7-4244-9893-873902d61202 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:16.160085 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6bffa24-38c7-4244-9893-873902d61202 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:16.160929 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6bffa24-38c7-4244-9893-873902d61202 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:16.161713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6bffa24-38c7-4244-9893-873902d61202 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:16.162502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6bffa24-38c7-4244-9893-873902d61202 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:16.162961 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6bffa24-38c7-4244-9893-873902d61202 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:16.163384 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6bffa24-38c7-4244-9893-873902d61202 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:16.196162 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c6bffa24-38c7-4244-9893-873902d61202 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058686 Sep 08 14:04:16.196751 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 861/1829] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:16 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:16.221685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f0c78281-0894-4f4a-85e9-9cb25eca0bba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:16.229826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f0c78281-0894-4f4a-85e9-9cb25eca0bba tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.193805 Sep 08 14:04:16.230432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 969/1830] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:16 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:16.791464 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-956aae25-13bc-4eae-9683-268ddfb15b76 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:16.794838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-956aae25-13bc-4eae-9683-268ddfb15b76 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:16.801894 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-956aae25-13bc-4eae-9683-268ddfb15b76 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:16.802423 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-956aae25-13bc-4eae-9683-268ddfb15b76 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:16.802867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-956aae25-13bc-4eae-9683-268ddfb15b76 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:16.803586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-956aae25-13bc-4eae-9683-268ddfb15b76 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:16.804042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-956aae25-13bc-4eae-9683-268ddfb15b76 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:16.804496 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-956aae25-13bc-4eae-9683-268ddfb15b76 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:16.840260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-956aae25-13bc-4eae-9683-268ddfb15b76 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052578 Sep 08 14:04:16.841387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 862/1831] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:16 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:16.865585 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7a0f29ba-134a-49f3-b826-1b8517ebc64c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:16.868444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7a0f29ba-134a-49f3-b826-1b8517ebc64c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:16.875766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7a0f29ba-134a-49f3-b826-1b8517ebc64c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:16.876261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7a0f29ba-134a-49f3-b826-1b8517ebc64c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:16.876701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7a0f29ba-134a-49f3-b826-1b8517ebc64c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:16.877417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7a0f29ba-134a-49f3-b826-1b8517ebc64c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:16.877903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7a0f29ba-134a-49f3-b826-1b8517ebc64c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:16.878346 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7a0f29ba-134a-49f3-b826-1b8517ebc64c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:16.909565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7a0f29ba-134a-49f3-b826-1b8517ebc64c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054332 Sep 08 14:04:16.910046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 970/1832] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:16 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:16.917037 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e7bf70e2-01f3-4fe1-babd-360da00a9dee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:16.917689 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e7bf70e2-01f3-4fe1-babd-360da00a9dee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:16.924465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7bf70e2-01f3-4fe1-babd-360da00a9dee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:16.924942 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7bf70e2-01f3-4fe1-babd-360da00a9dee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:16.925368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7bf70e2-01f3-4fe1-babd-360da00a9dee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:16.926058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7bf70e2-01f3-4fe1-babd-360da00a9dee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:16.926535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7bf70e2-01f3-4fe1-babd-360da00a9dee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:16.926955 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7bf70e2-01f3-4fe1-babd-360da00a9dee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:16.990552 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e7bf70e2-01f3-4fe1-babd-360da00a9dee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:16.999014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e7bf70e2-01f3-4fe1-babd-360da00a9dee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.086016 Sep 08 14:04:16.999600 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 863/1833] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:16 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:17.212924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5fd30117-9e87-4af4-9d4d-f564bb351a18 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:17.215178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5fd30117-9e87-4af4-9d4d-f564bb351a18 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:17.222057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5fd30117-9e87-4af4-9d4d-f564bb351a18 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:17.222582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5fd30117-9e87-4af4-9d4d-f564bb351a18 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:17.223031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5fd30117-9e87-4af4-9d4d-f564bb351a18 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:17.223739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5fd30117-9e87-4af4-9d4d-f564bb351a18 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:17.224201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5fd30117-9e87-4af4-9d4d-f564bb351a18 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:17.224635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5fd30117-9e87-4af4-9d4d-f564bb351a18 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:17.254893 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-00d222f8-11fc-43e4-8268-172aa9fc5f53 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:17.255499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-00d222f8-11fc-43e4-8268-172aa9fc5f53 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:17.256385 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5fd30117-9e87-4af4-9d4d-f564bb351a18 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050069 Sep 08 14:04:17.257154 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 971/1834] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:17 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:17.262786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00d222f8-11fc-43e4-8268-172aa9fc5f53 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:17.263292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00d222f8-11fc-43e4-8268-172aa9fc5f53 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:17.263733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00d222f8-11fc-43e4-8268-172aa9fc5f53 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:17.264452 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00d222f8-11fc-43e4-8268-172aa9fc5f53 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:17.264910 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00d222f8-11fc-43e4-8268-172aa9fc5f53 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:17.265332 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00d222f8-11fc-43e4-8268-172aa9fc5f53 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:17.403821 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-00d222f8-11fc-43e4-8268-172aa9fc5f53 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:17.412400 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-00d222f8-11fc-43e4-8268-172aa9fc5f53 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.161790 Sep 08 14:04:17.412958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 864/1835] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:17 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:17.855205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9945abd7-d024-44b2-b6d9-90ad1984c3af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:17.857475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9945abd7-d024-44b2-b6d9-90ad1984c3af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:17.866673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9945abd7-d024-44b2-b6d9-90ad1984c3af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:17.867356 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9945abd7-d024-44b2-b6d9-90ad1984c3af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:17.867604 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9945abd7-d024-44b2-b6d9-90ad1984c3af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:17.868386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9945abd7-d024-44b2-b6d9-90ad1984c3af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:17.869144 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9945abd7-d024-44b2-b6d9-90ad1984c3af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:17.869579 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9945abd7-d024-44b2-b6d9-90ad1984c3af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:17.906922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9945abd7-d024-44b2-b6d9-90ad1984c3af tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055925 Sep 08 14:04:17.907595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 972/1836] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:17 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:17.922834 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-85af82a2-b748-4f84-8d2e-c14088cbbe87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:17.925045 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-85af82a2-b748-4f84-8d2e-c14088cbbe87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:17.932567 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85af82a2-b748-4f84-8d2e-c14088cbbe87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:17.933107 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85af82a2-b748-4f84-8d2e-c14088cbbe87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:17.933487 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85af82a2-b748-4f84-8d2e-c14088cbbe87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:17.934796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85af82a2-b748-4f84-8d2e-c14088cbbe87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:17.935258 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85af82a2-b748-4f84-8d2e-c14088cbbe87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:17.935678 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-85af82a2-b748-4f84-8d2e-c14088cbbe87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:17.970477 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-85af82a2-b748-4f84-8d2e-c14088cbbe87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053138 Sep 08 14:04:17.971085 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 865/1837] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:17 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:18.026414 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-17e6ad18-1084-4e02-97c2-275c6516e6fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:18.027041 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-17e6ad18-1084-4e02-97c2-275c6516e6fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:18.033957 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-17e6ad18-1084-4e02-97c2-275c6516e6fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:18.034527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-17e6ad18-1084-4e02-97c2-275c6516e6fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:18.034971 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-17e6ad18-1084-4e02-97c2-275c6516e6fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:18.035927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-17e6ad18-1084-4e02-97c2-275c6516e6fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:18.036388 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-17e6ad18-1084-4e02-97c2-275c6516e6fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:18.036821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-17e6ad18-1084-4e02-97c2-275c6516e6fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:18.103741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-17e6ad18-1084-4e02-97c2-275c6516e6fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:18.112141 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-17e6ad18-1084-4e02-97c2-275c6516e6fc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.091509 Sep 08 14:04:18.112635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 973/1838] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:18 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:18.269995 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7e450903-fdc5-4c7c-bdda-63fe8bfb90d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:18.272303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7e450903-fdc5-4c7c-bdda-63fe8bfb90d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:18.274311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:18.279145 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e450903-fdc5-4c7c-bdda-63fe8bfb90d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:18.279634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e450903-fdc5-4c7c-bdda-63fe8bfb90d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:18.280073 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e450903-fdc5-4c7c-bdda-63fe8bfb90d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:18.280782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e450903-fdc5-4c7c-bdda-63fe8bfb90d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:18.281239 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e450903-fdc5-4c7c-bdda-63fe8bfb90d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:18.281668 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e450903-fdc5-4c7c-bdda-63fe8bfb90d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:18.318821 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7e450903-fdc5-4c7c-bdda-63fe8bfb90d7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052815 Sep 08 14:04:18.319419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 866/1839] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:18 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:18.447030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0e5abb18-8a15-470f-9808-71a015f766e0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:18.448045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0e5abb18-8a15-470f-9808-71a015f766e0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:18.457392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e5abb18-8a15-470f-9808-71a015f766e0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:18.458026 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e5abb18-8a15-470f-9808-71a015f766e0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:18.458608 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e5abb18-8a15-470f-9808-71a015f766e0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:18.459453 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e5abb18-8a15-470f-9808-71a015f766e0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:18.460078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e5abb18-8a15-470f-9808-71a015f766e0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:18.460728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e5abb18-8a15-470f-9808-71a015f766e0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:18.628231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0e5abb18-8a15-470f-9808-71a015f766e0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:18.638693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0e5abb18-8a15-470f-9808-71a015f766e0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.196121 Sep 08 14:04:18.639280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 974/1840] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:18 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:18.920785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-528da9d2-2a56-45e3-80a5-f020ac147827 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:18.923225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-528da9d2-2a56-45e3-80a5-f020ac147827 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:18.929956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-528da9d2-2a56-45e3-80a5-f020ac147827 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:18.930470 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-528da9d2-2a56-45e3-80a5-f020ac147827 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:18.930894 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-528da9d2-2a56-45e3-80a5-f020ac147827 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:18.931589 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-528da9d2-2a56-45e3-80a5-f020ac147827 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:18.932065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-528da9d2-2a56-45e3-80a5-f020ac147827 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:18.932462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-528da9d2-2a56-45e3-80a5-f020ac147827 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:18.970761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-528da9d2-2a56-45e3-80a5-f020ac147827 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054057 Sep 08 14:04:18.971349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 867/1841] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:18 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:18.983421 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0075df20-758d-4387-be34-b8eae4c434f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:18.985624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0075df20-758d-4387-be34-b8eae4c434f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:18.992233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0075df20-758d-4387-be34-b8eae4c434f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:18.992716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0075df20-758d-4387-be34-b8eae4c434f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:18.993159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0075df20-758d-4387-be34-b8eae4c434f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:18.994118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0075df20-758d-4387-be34-b8eae4c434f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:18.994667 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0075df20-758d-4387-be34-b8eae4c434f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:18.995101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0075df20-758d-4387-be34-b8eae4c434f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:19.026648 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0075df20-758d-4387-be34-b8eae4c434f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047275 Sep 08 14:04:19.027219 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 975/1842] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:18 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:19.144103 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7142d6a6-dce7-4b86-84dc-3332f5e4c324 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:19.144103 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7142d6a6-dce7-4b86-84dc-3332f5e4c324 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:19.149332 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7142d6a6-dce7-4b86-84dc-3332f5e4c324 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:19.150188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7142d6a6-dce7-4b86-84dc-3332f5e4c324 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:19.150969 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7142d6a6-dce7-4b86-84dc-3332f5e4c324 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:19.152115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7142d6a6-dce7-4b86-84dc-3332f5e4c324 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:19.152591 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7142d6a6-dce7-4b86-84dc-3332f5e4c324 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:19.153280 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7142d6a6-dce7-4b86-84dc-3332f5e4c324 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:19.242258 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7142d6a6-dce7-4b86-84dc-3332f5e4c324 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:19.254913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7142d6a6-dce7-4b86-84dc-3332f5e4c324 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.119058 Sep 08 14:04:19.255789 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 868/1843] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:19 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:19.333681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-22dd8cbe-a782-47e0-b94f-0292b209c720 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:19.334945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-22dd8cbe-a782-47e0-b94f-0292b209c720 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:19.341931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22dd8cbe-a782-47e0-b94f-0292b209c720 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:19.342646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22dd8cbe-a782-47e0-b94f-0292b209c720 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:19.343198 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22dd8cbe-a782-47e0-b94f-0292b209c720 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:19.344009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22dd8cbe-a782-47e0-b94f-0292b209c720 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:19.344557 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22dd8cbe-a782-47e0-b94f-0292b209c720 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:19.345095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22dd8cbe-a782-47e0-b94f-0292b209c720 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:19.348047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:19.380001 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-22dd8cbe-a782-47e0-b94f-0292b209c720 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051272 Sep 08 14:04:19.380591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 976/1844] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:19 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:19.665529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-56024bdf-5e3e-4212-b6f8-3d2205990f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:19.666553 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-56024bdf-5e3e-4212-b6f8-3d2205990f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:19.677112 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-56024bdf-5e3e-4212-b6f8-3d2205990f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:19.677767 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-56024bdf-5e3e-4212-b6f8-3d2205990f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:19.678351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-56024bdf-5e3e-4212-b6f8-3d2205990f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:19.679369 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-56024bdf-5e3e-4212-b6f8-3d2205990f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:19.680152 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-56024bdf-5e3e-4212-b6f8-3d2205990f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:19.680912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-56024bdf-5e3e-4212-b6f8-3d2205990f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:19.849852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-56024bdf-5e3e-4212-b6f8-3d2205990f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:19.861214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-56024bdf-5e3e-4212-b6f8-3d2205990f8f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.202215 Sep 08 14:04:19.861803 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 869/1845] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:19 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:19.984524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-43342028-bc66-4d41-a0b3-e1aba03acff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:19.987189 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-43342028-bc66-4d41-a0b3-e1aba03acff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:19.994621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43342028-bc66-4d41-a0b3-e1aba03acff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:19.995118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43342028-bc66-4d41-a0b3-e1aba03acff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:19.995556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43342028-bc66-4d41-a0b3-e1aba03acff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:19.996259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43342028-bc66-4d41-a0b3-e1aba03acff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:19.997226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43342028-bc66-4d41-a0b3-e1aba03acff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:19.997655 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43342028-bc66-4d41-a0b3-e1aba03acff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:20.030354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-43342028-bc66-4d41-a0b3-e1aba03acff4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050153 Sep 08 14:04:20.030931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 977/1846] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:19 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:20.040773 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-390878bd-0ac1-4505-9f2b-89031a83ed02 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:20.043073 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-390878bd-0ac1-4505-9f2b-89031a83ed02 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:20.049483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-390878bd-0ac1-4505-9f2b-89031a83ed02 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:20.050038 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-390878bd-0ac1-4505-9f2b-89031a83ed02 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:20.050522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-390878bd-0ac1-4505-9f2b-89031a83ed02 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:20.051221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-390878bd-0ac1-4505-9f2b-89031a83ed02 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:20.051666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-390878bd-0ac1-4505-9f2b-89031a83ed02 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:20.052084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-390878bd-0ac1-4505-9f2b-89031a83ed02 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:20.084169 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-390878bd-0ac1-4505-9f2b-89031a83ed02 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048086 Sep 08 14:04:20.084927 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 870/1847] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:20 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:20.280047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1a9a61a5-763f-41ba-96b2-dbc0a1d3200d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:20.280565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1a9a61a5-763f-41ba-96b2-dbc0a1d3200d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:20.286582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1a9a61a5-763f-41ba-96b2-dbc0a1d3200d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:20.287066 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1a9a61a5-763f-41ba-96b2-dbc0a1d3200d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:20.287532 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1a9a61a5-763f-41ba-96b2-dbc0a1d3200d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:20.288231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1a9a61a5-763f-41ba-96b2-dbc0a1d3200d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:20.288684 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1a9a61a5-763f-41ba-96b2-dbc0a1d3200d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:20.289106 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1a9a61a5-763f-41ba-96b2-dbc0a1d3200d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:20.352192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1a9a61a5-763f-41ba-96b2-dbc0a1d3200d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:20.360901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1a9a61a5-763f-41ba-96b2-dbc0a1d3200d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.084779 Sep 08 14:04:20.361477 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 978/1848] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:20 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:20.393539 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1c4af7d1-4ebf-4f2d-8136-22dbcfe3a948 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:20.395775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1c4af7d1-4ebf-4f2d-8136-22dbcfe3a948 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:20.409392 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c4af7d1-4ebf-4f2d-8136-22dbcfe3a948 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:20.409881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c4af7d1-4ebf-4f2d-8136-22dbcfe3a948 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:20.410357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c4af7d1-4ebf-4f2d-8136-22dbcfe3a948 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:20.411067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c4af7d1-4ebf-4f2d-8136-22dbcfe3a948 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:20.411520 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c4af7d1-4ebf-4f2d-8136-22dbcfe3a948 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:20.411953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1c4af7d1-4ebf-4f2d-8136-22dbcfe3a948 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:20.444179 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1c4af7d1-4ebf-4f2d-8136-22dbcfe3a948 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054582 Sep 08 14:04:20.444750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 871/1849] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:20 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:20.885586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bfafc8ad-075c-4f48-b7e6-582c0641ebab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:20.886171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bfafc8ad-075c-4f48-b7e6-582c0641ebab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:20.893047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bfafc8ad-075c-4f48-b7e6-582c0641ebab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:20.893473 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bfafc8ad-075c-4f48-b7e6-582c0641ebab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:20.893908 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bfafc8ad-075c-4f48-b7e6-582c0641ebab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:20.894646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bfafc8ad-075c-4f48-b7e6-582c0641ebab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:20.895104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bfafc8ad-075c-4f48-b7e6-582c0641ebab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:20.895526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bfafc8ad-075c-4f48-b7e6-582c0641ebab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:21.043584 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c81dbc30-1d88-4dae-a62b-c47d529999d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:21.045738 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bfafc8ad-075c-4f48-b7e6-582c0641ebab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:21.046178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c81dbc30-1d88-4dae-a62b-c47d529999d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:21.052676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c81dbc30-1d88-4dae-a62b-c47d529999d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:21.053168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c81dbc30-1d88-4dae-a62b-c47d529999d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:21.053605 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c81dbc30-1d88-4dae-a62b-c47d529999d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:21.054348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c81dbc30-1d88-4dae-a62b-c47d529999d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:21.054799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c81dbc30-1d88-4dae-a62b-c47d529999d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:21.055223 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c81dbc30-1d88-4dae-a62b-c47d529999d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:21.058636 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bfafc8ad-075c-4f48-b7e6-582c0641ebab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.176837 Sep 08 14:04:21.059197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:21.064879 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 979/1850] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:20 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:21.088007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c81dbc30-1d88-4dae-a62b-c47d529999d7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048812 Sep 08 14:04:21.088594 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 872/1851] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:21 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:21.099141 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9149644e-e907-4d71-b575-638ece8f7aa6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:21.101845 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9149644e-e907-4d71-b575-638ece8f7aa6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:21.109328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9149644e-e907-4d71-b575-638ece8f7aa6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:21.110365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9149644e-e907-4d71-b575-638ece8f7aa6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:21.110499 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9149644e-e907-4d71-b575-638ece8f7aa6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:21.111511 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9149644e-e907-4d71-b575-638ece8f7aa6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:21.112107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9149644e-e907-4d71-b575-638ece8f7aa6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:21.112540 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9149644e-e907-4d71-b575-638ece8f7aa6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:21.147135 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9149644e-e907-4d71-b575-638ece8f7aa6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052305 Sep 08 14:04:21.147846 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 980/1852] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:21 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:21.385407 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4a866627-257d-455a-9066-7f2dc7074b26 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:21.386072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4a866627-257d-455a-9066-7f2dc7074b26 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:21.393607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a866627-257d-455a-9066-7f2dc7074b26 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:21.394094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a866627-257d-455a-9066-7f2dc7074b26 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:21.394562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a866627-257d-455a-9066-7f2dc7074b26 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:21.395265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a866627-257d-455a-9066-7f2dc7074b26 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:21.395717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a866627-257d-455a-9066-7f2dc7074b26 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:21.396184 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a866627-257d-455a-9066-7f2dc7074b26 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:21.457525 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4b688ef6-6d66-4831-9a00-5dc25e5492d8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:21.459736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4b688ef6-6d66-4831-9a00-5dc25e5492d8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:21.467153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4b688ef6-6d66-4831-9a00-5dc25e5492d8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:21.467645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4b688ef6-6d66-4831-9a00-5dc25e5492d8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:21.468282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4b688ef6-6d66-4831-9a00-5dc25e5492d8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:21.468485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4a866627-257d-455a-9066-7f2dc7074b26 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:21.471226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4b688ef6-6d66-4831-9a00-5dc25e5492d8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:21.471226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4b688ef6-6d66-4831-9a00-5dc25e5492d8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:21.471226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4b688ef6-6d66-4831-9a00-5dc25e5492d8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:21.479252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4a866627-257d-455a-9066-7f2dc7074b26 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.098010 Sep 08 14:04:21.479864 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 873/1853] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:21 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:21.505098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4b688ef6-6d66-4831-9a00-5dc25e5492d8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051887 Sep 08 14:04:21.505671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 981/1854] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:21 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:04:22.087570 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a0040cd6-425f-4060-b6ad-2fa74e547426 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:22.088232 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a0040cd6-425f-4060-b6ad-2fa74e547426 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:22.095086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0040cd6-425f-4060-b6ad-2fa74e547426 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:22.095568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0040cd6-425f-4060-b6ad-2fa74e547426 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:22.096003 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0040cd6-425f-4060-b6ad-2fa74e547426 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:22.096698 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0040cd6-425f-4060-b6ad-2fa74e547426 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:22.097154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0040cd6-425f-4060-b6ad-2fa74e547426 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:22.097573 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a0040cd6-425f-4060-b6ad-2fa74e547426 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:22.102401 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8f9e43cb-89fa-4296-8baa-5fa8dad52528 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:22.104551 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8f9e43cb-89fa-4296-8baa-5fa8dad52528 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:22.111002 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8f9e43cb-89fa-4296-8baa-5fa8dad52528 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:22.111486 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8f9e43cb-89fa-4296-8baa-5fa8dad52528 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:22.111920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8f9e43cb-89fa-4296-8baa-5fa8dad52528 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:22.112626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8f9e43cb-89fa-4296-8baa-5fa8dad52528 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:22.113074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8f9e43cb-89fa-4296-8baa-5fa8dad52528 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:22.113498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8f9e43cb-89fa-4296-8baa-5fa8dad52528 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:22.149342 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8f9e43cb-89fa-4296-8baa-5fa8dad52528 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051090 Sep 08 14:04:22.149836 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 982/1855] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:22 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:22.160039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3bebdc98-27af-4c8f-9f06-dccd08ccee49 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:22.163023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3bebdc98-27af-4c8f-9f06-dccd08ccee49 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:22.168625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3bebdc98-27af-4c8f-9f06-dccd08ccee49 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:22.169115 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3bebdc98-27af-4c8f-9f06-dccd08ccee49 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:22.169553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3bebdc98-27af-4c8f-9f06-dccd08ccee49 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:22.170288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3bebdc98-27af-4c8f-9f06-dccd08ccee49 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:22.171680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3bebdc98-27af-4c8f-9f06-dccd08ccee49 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:22.171680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3bebdc98-27af-4c8f-9f06-dccd08ccee49 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:22.203916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3bebdc98-27af-4c8f-9f06-dccd08ccee49 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047747 Sep 08 14:04:22.204997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 983/1856] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:22 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:22.257391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a0040cd6-425f-4060-b6ad-2fa74e547426 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:22.265947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a0040cd6-425f-4060-b6ad-2fa74e547426 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.182237 Sep 08 14:04:22.266549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 874/1857] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:22 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:22.503776 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c2bb56e7-49b9-49e7-a29c-aee0c2d70aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:22.504315 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c2bb56e7-49b9-49e7-a29c-aee0c2d70aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:22.510882 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2bb56e7-49b9-49e7-a29c-aee0c2d70aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:22.511389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2bb56e7-49b9-49e7-a29c-aee0c2d70aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:22.511832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2bb56e7-49b9-49e7-a29c-aee0c2d70aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:22.512546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2bb56e7-49b9-49e7-a29c-aee0c2d70aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:22.513002 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2bb56e7-49b9-49e7-a29c-aee0c2d70aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:22.513436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2bb56e7-49b9-49e7-a29c-aee0c2d70aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:22.518404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0efd994d-9f60-44e8-9ab3-0d54d803d401 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:22.520621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0efd994d-9f60-44e8-9ab3-0d54d803d401 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:22.526998 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0efd994d-9f60-44e8-9ab3-0d54d803d401 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:22.527496 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0efd994d-9f60-44e8-9ab3-0d54d803d401 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:22.527966 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0efd994d-9f60-44e8-9ab3-0d54d803d401 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:22.528720 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0efd994d-9f60-44e8-9ab3-0d54d803d401 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:22.529221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0efd994d-9f60-44e8-9ab3-0d54d803d401 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:22.529717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0efd994d-9f60-44e8-9ab3-0d54d803d401 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:22.573912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0efd994d-9f60-44e8-9ab3-0d54d803d401 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.059776 Sep 08 14:04:22.574839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 875/1858] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:22 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:04:22.582411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c2bb56e7-49b9-49e7-a29c-aee0c2d70aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:22.594466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c2bb56e7-49b9-49e7-a29c-aee0c2d70aae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.094636 Sep 08 14:04:22.595055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 984/1859] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:22 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:23.166034 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-eb76d415-6a6b-4e96-903a-67c4b2839e90 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:23.168260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-eb76d415-6a6b-4e96-903a-67c4b2839e90 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:23.175233 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb76d415-6a6b-4e96-903a-67c4b2839e90 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:23.175717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb76d415-6a6b-4e96-903a-67c4b2839e90 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:23.176151 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb76d415-6a6b-4e96-903a-67c4b2839e90 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:23.176852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb76d415-6a6b-4e96-903a-67c4b2839e90 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:23.177303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb76d415-6a6b-4e96-903a-67c4b2839e90 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:23.177716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb76d415-6a6b-4e96-903a-67c4b2839e90 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:23.210327 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-eb76d415-6a6b-4e96-903a-67c4b2839e90 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048602 Sep 08 14:04:23.210907 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 876/1860] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:23 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:23.217108 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2662f0c2-268e-4e71-a152-c3b296aa0c12 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:23.219452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2662f0c2-268e-4e71-a152-c3b296aa0c12 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:23.225974 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2662f0c2-268e-4e71-a152-c3b296aa0c12 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:23.226497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2662f0c2-268e-4e71-a152-c3b296aa0c12 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:23.226932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2662f0c2-268e-4e71-a152-c3b296aa0c12 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:23.227624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2662f0c2-268e-4e71-a152-c3b296aa0c12 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:23.228060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2662f0c2-268e-4e71-a152-c3b296aa0c12 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:23.228482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2662f0c2-268e-4e71-a152-c3b296aa0c12 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:23.261081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2662f0c2-268e-4e71-a152-c3b296aa0c12 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048468 Sep 08 14:04:23.262233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 985/1861] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:23 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:23.289633 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ec141381-0088-4605-be90-be93322f2aff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:23.290267 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ec141381-0088-4605-be90-be93322f2aff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:23.297316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ec141381-0088-4605-be90-be93322f2aff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:23.297797 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ec141381-0088-4605-be90-be93322f2aff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:23.298396 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ec141381-0088-4605-be90-be93322f2aff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:23.299050 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ec141381-0088-4605-be90-be93322f2aff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:23.299513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ec141381-0088-4605-be90-be93322f2aff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:23.299931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ec141381-0088-4605-be90-be93322f2aff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:23.434975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ec141381-0088-4605-be90-be93322f2aff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:23.443028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ec141381-0088-4605-be90-be93322f2aff tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.157251 Sep 08 14:04:23.443613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 877/1862] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:23 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:23.589933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c742eed6-347c-4082-9173-c1ff29f77c5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:23.593228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c742eed6-347c-4082-9173-c1ff29f77c5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:23.600997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c742eed6-347c-4082-9173-c1ff29f77c5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:23.601482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c742eed6-347c-4082-9173-c1ff29f77c5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:23.601923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c742eed6-347c-4082-9173-c1ff29f77c5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:23.602672 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c742eed6-347c-4082-9173-c1ff29f77c5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:23.603127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c742eed6-347c-4082-9173-c1ff29f77c5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:23.603553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c742eed6-347c-4082-9173-c1ff29f77c5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:23.618446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-43010869-4604-41f0-aa42-98e7a261e126 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:23.619572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-43010869-4604-41f0-aa42-98e7a261e126 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:23.626006 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-43010869-4604-41f0-aa42-98e7a261e126 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:23.626782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-43010869-4604-41f0-aa42-98e7a261e126 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:23.627374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-43010869-4604-41f0-aa42-98e7a261e126 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:23.628256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-43010869-4604-41f0-aa42-98e7a261e126 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:23.628712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-43010869-4604-41f0-aa42-98e7a261e126 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:23.629134 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-43010869-4604-41f0-aa42-98e7a261e126 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:23.636265 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c742eed6-347c-4082-9173-c1ff29f77c5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050620 Sep 08 14:04:23.637629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 986/1863] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:23 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:23.692432 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-43010869-4604-41f0-aa42-98e7a261e126 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:23.701164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-43010869-4604-41f0-aa42-98e7a261e126 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.086437 Sep 08 14:04:23.701687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 878/1864] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:23 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:24.223899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5da7401c-df2e-42a0-a0f1-70871bf7bdc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:24.225925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5da7401c-df2e-42a0-a0f1-70871bf7bdc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:24.233467 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5da7401c-df2e-42a0-a0f1-70871bf7bdc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:24.233943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5da7401c-df2e-42a0-a0f1-70871bf7bdc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:24.234400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5da7401c-df2e-42a0-a0f1-70871bf7bdc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:24.235157 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5da7401c-df2e-42a0-a0f1-70871bf7bdc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:24.236072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5da7401c-df2e-42a0-a0f1-70871bf7bdc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:24.236072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5da7401c-df2e-42a0-a0f1-70871bf7bdc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:24.269004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5da7401c-df2e-42a0-a0f1-70871bf7bdc7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049137 Sep 08 14:04:24.269585 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 987/1865] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:24 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:24.275251 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c6966557-883e-4230-af68-edea46c7fcaf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:24.277911 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c6966557-883e-4230-af68-edea46c7fcaf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:24.285182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6966557-883e-4230-af68-edea46c7fcaf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:24.285666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6966557-883e-4230-af68-edea46c7fcaf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:24.286094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6966557-883e-4230-af68-edea46c7fcaf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:24.286849 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6966557-883e-4230-af68-edea46c7fcaf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:24.287291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6966557-883e-4230-af68-edea46c7fcaf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:24.287703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6966557-883e-4230-af68-edea46c7fcaf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:24.320494 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c6966557-883e-4230-af68-edea46c7fcaf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049290 Sep 08 14:04:24.321386 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 879/1866] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:24 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:24.467437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0987e1c2-b49a-4659-9b50-2f6926dc8f0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:24.468082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0987e1c2-b49a-4659-9b50-2f6926dc8f0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:24.474353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0987e1c2-b49a-4659-9b50-2f6926dc8f0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:24.474776 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0987e1c2-b49a-4659-9b50-2f6926dc8f0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:24.475216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0987e1c2-b49a-4659-9b50-2f6926dc8f0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:24.475918 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0987e1c2-b49a-4659-9b50-2f6926dc8f0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:24.476374 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0987e1c2-b49a-4659-9b50-2f6926dc8f0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:24.476800 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0987e1c2-b49a-4659-9b50-2f6926dc8f0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:24.624614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0987e1c2-b49a-4659-9b50-2f6926dc8f0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:24.632762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0987e1c2-b49a-4659-9b50-2f6926dc8f0d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.169847 Sep 08 14:04:24.633548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 988/1867] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:24 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:24.655036 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7c8af7d5-7a74-4a68-ad15-88639fd2cc99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:24.658730 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7c8af7d5-7a74-4a68-ad15-88639fd2cc99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:24.669484 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c8af7d5-7a74-4a68-ad15-88639fd2cc99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:24.670364 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c8af7d5-7a74-4a68-ad15-88639fd2cc99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:24.675499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c8af7d5-7a74-4a68-ad15-88639fd2cc99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:24.675499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c8af7d5-7a74-4a68-ad15-88639fd2cc99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:24.675499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c8af7d5-7a74-4a68-ad15-88639fd2cc99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:24.675499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c8af7d5-7a74-4a68-ad15-88639fd2cc99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:24.707649 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7c8af7d5-7a74-4a68-ad15-88639fd2cc99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061170 Sep 08 14:04:24.708216 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 880/1868] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:24 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:24.736187 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1d2880d5-1abb-4ea1-85b4-ccfeaa1ec36c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:24.736693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1d2880d5-1abb-4ea1-85b4-ccfeaa1ec36c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:24.743398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d2880d5-1abb-4ea1-85b4-ccfeaa1ec36c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:24.744145 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d2880d5-1abb-4ea1-85b4-ccfeaa1ec36c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:24.744587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d2880d5-1abb-4ea1-85b4-ccfeaa1ec36c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:24.745704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d2880d5-1abb-4ea1-85b4-ccfeaa1ec36c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:24.746003 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d2880d5-1abb-4ea1-85b4-ccfeaa1ec36c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:24.746462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d2880d5-1abb-4ea1-85b4-ccfeaa1ec36c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:24.819442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1d2880d5-1abb-4ea1-85b4-ccfeaa1ec36c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:24.829364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1d2880d5-1abb-4ea1-85b4-ccfeaa1ec36c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.097223 Sep 08 14:04:24.829944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 989/1869] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:24 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:25.281268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1429f761-c1b1-4416-a4f0-8b38b3c90f0c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:25.283488 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1429f761-c1b1-4416-a4f0-8b38b3c90f0c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:25.290688 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1429f761-c1b1-4416-a4f0-8b38b3c90f0c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:25.291170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1429f761-c1b1-4416-a4f0-8b38b3c90f0c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:25.291595 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1429f761-c1b1-4416-a4f0-8b38b3c90f0c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:25.292289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1429f761-c1b1-4416-a4f0-8b38b3c90f0c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:25.292780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1429f761-c1b1-4416-a4f0-8b38b3c90f0c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:25.293174 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1429f761-c1b1-4416-a4f0-8b38b3c90f0c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:25.328249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1429f761-c1b1-4416-a4f0-8b38b3c90f0c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050977 Sep 08 14:04:25.328823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 881/1870] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:25 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:25.336716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2a08d778-efa8-4891-a759-070fc1a390dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:25.339269 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2a08d778-efa8-4891-a759-070fc1a390dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:25.341243 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:25.346618 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a08d778-efa8-4891-a759-070fc1a390dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:25.347238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a08d778-efa8-4891-a759-070fc1a390dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:25.347780 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a08d778-efa8-4891-a759-070fc1a390dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:25.348591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a08d778-efa8-4891-a759-070fc1a390dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:25.349145 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a08d778-efa8-4891-a759-070fc1a390dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:25.349693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2a08d778-efa8-4891-a759-070fc1a390dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:25.388075 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2a08d778-efa8-4891-a759-070fc1a390dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058471 Sep 08 14:04:25.388662 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 990/1871] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:25 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:25.665452 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-865f0ff2-0ed7-4843-b5f3-1637690b81e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:25.666633 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-865f0ff2-0ed7-4843-b5f3-1637690b81e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:25.676096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-865f0ff2-0ed7-4843-b5f3-1637690b81e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:25.676591 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-865f0ff2-0ed7-4843-b5f3-1637690b81e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:25.677030 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-865f0ff2-0ed7-4843-b5f3-1637690b81e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:25.677745 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-865f0ff2-0ed7-4843-b5f3-1637690b81e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:25.678224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-865f0ff2-0ed7-4843-b5f3-1637690b81e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:25.678648 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-865f0ff2-0ed7-4843-b5f3-1637690b81e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:25.722970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c2af827b-6369-489b-a06b-01bde94981b9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:25.726481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c2af827b-6369-489b-a06b-01bde94981b9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:25.734330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2af827b-6369-489b-a06b-01bde94981b9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:25.734828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2af827b-6369-489b-a06b-01bde94981b9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:25.735274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2af827b-6369-489b-a06b-01bde94981b9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:25.736016 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2af827b-6369-489b-a06b-01bde94981b9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:25.736517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2af827b-6369-489b-a06b-01bde94981b9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:25.736948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2af827b-6369-489b-a06b-01bde94981b9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:25.784593 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c2af827b-6369-489b-a06b-01bde94981b9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.067809 Sep 08 14:04:25.785181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 991/1872] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:25 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:04:25.852017 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-865f0ff2-0ed7-4843-b5f3-1637690b81e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:25.854631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6451038d-cb4e-443d-a1f9-f06671c2f2eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:25.855314 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6451038d-cb4e-443d-a1f9-f06671c2f2eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:25.862466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6451038d-cb4e-443d-a1f9-f06671c2f2eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:25.863146 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6451038d-cb4e-443d-a1f9-f06671c2f2eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:25.863627 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6451038d-cb4e-443d-a1f9-f06671c2f2eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:25.864419 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6451038d-cb4e-443d-a1f9-f06671c2f2eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:25.864884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6451038d-cb4e-443d-a1f9-f06671c2f2eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:25.866706 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6451038d-cb4e-443d-a1f9-f06671c2f2eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:25.871939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-865f0ff2-0ed7-4843-b5f3-1637690b81e5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.212519 Sep 08 14:04:25.872797 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 882/1873] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:25 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:25.961336 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6451038d-cb4e-443d-a1f9-f06671c2f2eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:25.964194 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:25.972647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6451038d-cb4e-443d-a1f9-f06671c2f2eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.122490 Sep 08 14:04:25.976057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 992/1874] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:25 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:26.344610 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-65e51398-cfe2-43a1-9b44-143774cbc720 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:26.346780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-65e51398-cfe2-43a1-9b44-143774cbc720 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:26.353512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-65e51398-cfe2-43a1-9b44-143774cbc720 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:26.353995 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-65e51398-cfe2-43a1-9b44-143774cbc720 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:26.354461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-65e51398-cfe2-43a1-9b44-143774cbc720 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:26.355166 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-65e51398-cfe2-43a1-9b44-143774cbc720 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:26.355611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-65e51398-cfe2-43a1-9b44-143774cbc720 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:26.356036 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-65e51398-cfe2-43a1-9b44-143774cbc720 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:26.392329 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-65e51398-cfe2-43a1-9b44-143774cbc720 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052358 Sep 08 14:04:26.393160 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 883/1875] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:26 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:26.401777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-29a4da18-0b1c-4718-826b-88e08e6c857b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:26.405182 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-29a4da18-0b1c-4718-826b-88e08e6c857b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:26.412973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-29a4da18-0b1c-4718-826b-88e08e6c857b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:26.413479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-29a4da18-0b1c-4718-826b-88e08e6c857b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:26.414070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-29a4da18-0b1c-4718-826b-88e08e6c857b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:26.415303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-29a4da18-0b1c-4718-826b-88e08e6c857b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:26.415930 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-29a4da18-0b1c-4718-826b-88e08e6c857b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:26.416523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-29a4da18-0b1c-4718-826b-88e08e6c857b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:26.453024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-29a4da18-0b1c-4718-826b-88e08e6c857b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055952 Sep 08 14:04:26.466252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 993/1876] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:26 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:26.798296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-da2ecd20-fc87-40ce-ab4f-b32a2fe0fdba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:26.800453 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-da2ecd20-fc87-40ce-ab4f-b32a2fe0fdba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:26.806718 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da2ecd20-fc87-40ce-ab4f-b32a2fe0fdba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:26.807201 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da2ecd20-fc87-40ce-ab4f-b32a2fe0fdba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:26.807632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da2ecd20-fc87-40ce-ab4f-b32a2fe0fdba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:26.808858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da2ecd20-fc87-40ce-ab4f-b32a2fe0fdba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:26.809302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da2ecd20-fc87-40ce-ab4f-b32a2fe0fdba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:26.809716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da2ecd20-fc87-40ce-ab4f-b32a2fe0fdba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:26.848449 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-da2ecd20-fc87-40ce-ab4f-b32a2fe0fdba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054256 Sep 08 14:04:26.849215 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 884/1877] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:26 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:26.899453 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0e7f3142-ca17-4a47-a199-95b8a6a4739a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:26.900534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0e7f3142-ca17-4a47-a199-95b8a6a4739a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:26.907914 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e7f3142-ca17-4a47-a199-95b8a6a4739a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:26.908401 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e7f3142-ca17-4a47-a199-95b8a6a4739a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:26.908838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e7f3142-ca17-4a47-a199-95b8a6a4739a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:26.909536 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e7f3142-ca17-4a47-a199-95b8a6a4739a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:26.909989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e7f3142-ca17-4a47-a199-95b8a6a4739a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:26.910472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0e7f3142-ca17-4a47-a199-95b8a6a4739a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:26.999253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ed5f40cf-e8e7-4069-be83-8baf6ae43137 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:27.000054 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ed5f40cf-e8e7-4069-be83-8baf6ae43137 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:27.007081 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed5f40cf-e8e7-4069-be83-8baf6ae43137 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:27.007566 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed5f40cf-e8e7-4069-be83-8baf6ae43137 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:27.008057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed5f40cf-e8e7-4069-be83-8baf6ae43137 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:27.008927 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed5f40cf-e8e7-4069-be83-8baf6ae43137 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:27.009683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed5f40cf-e8e7-4069-be83-8baf6ae43137 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:27.010322 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed5f40cf-e8e7-4069-be83-8baf6ae43137 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:27.061549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0e7f3142-ca17-4a47-a199-95b8a6a4739a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:27.069366 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0e7f3142-ca17-4a47-a199-95b8a6a4739a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.177124 Sep 08 14:04:27.069937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 994/1878] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:26 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:27.075758 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ed5f40cf-e8e7-4069-be83-8baf6ae43137 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:27.088051 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ed5f40cf-e8e7-4069-be83-8baf6ae43137 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.094437 Sep 08 14:04:27.088741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 885/1879] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:26 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:27.405431 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-706ecf18-be77-4339-90e2-bd3643f1f30b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:27.407700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-706ecf18-be77-4339-90e2-bd3643f1f30b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:27.414602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-706ecf18-be77-4339-90e2-bd3643f1f30b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:27.415108 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-706ecf18-be77-4339-90e2-bd3643f1f30b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:27.415554 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-706ecf18-be77-4339-90e2-bd3643f1f30b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:27.416285 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-706ecf18-be77-4339-90e2-bd3643f1f30b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:27.416833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-706ecf18-be77-4339-90e2-bd3643f1f30b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:27.417276 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-706ecf18-be77-4339-90e2-bd3643f1f30b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:27.456740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-706ecf18-be77-4339-90e2-bd3643f1f30b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055856 Sep 08 14:04:27.457333 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 995/1880] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:27 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:27.486612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a78e328d-01e8-44f0-ab09-3c368ebfed9c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:27.489047 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a78e328d-01e8-44f0-ab09-3c368ebfed9c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:27.497414 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a78e328d-01e8-44f0-ab09-3c368ebfed9c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:27.498034 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a78e328d-01e8-44f0-ab09-3c368ebfed9c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:27.498556 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a78e328d-01e8-44f0-ab09-3c368ebfed9c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:27.499741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a78e328d-01e8-44f0-ab09-3c368ebfed9c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:27.500313 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a78e328d-01e8-44f0-ab09-3c368ebfed9c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:27.500839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a78e328d-01e8-44f0-ab09-3c368ebfed9c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:27.538786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a78e328d-01e8-44f0-ab09-3c368ebfed9c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064601 Sep 08 14:04:27.539382 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 886/1881] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:27 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:27.862504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e3450c20-6c48-4bb8-8cbd-657f254f3cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:27.864685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e3450c20-6c48-4bb8-8cbd-657f254f3cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:27.871156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e3450c20-6c48-4bb8-8cbd-657f254f3cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:27.871647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e3450c20-6c48-4bb8-8cbd-657f254f3cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:27.872085 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e3450c20-6c48-4bb8-8cbd-657f254f3cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:27.872797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e3450c20-6c48-4bb8-8cbd-657f254f3cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:27.873299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e3450c20-6c48-4bb8-8cbd-657f254f3cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:27.873724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e3450c20-6c48-4bb8-8cbd-657f254f3cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:27.908851 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e3450c20-6c48-4bb8-8cbd-657f254f3cb0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050795 Sep 08 14:04:27.909435 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 996/1882] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:27 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:28.095458 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cf6612b7-1dd8-468a-9c6c-6825e76e4bcc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:28.096610 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cf6612b7-1dd8-468a-9c6c-6825e76e4bcc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:28.105135 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cf6612b7-1dd8-468a-9c6c-6825e76e4bcc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:28.105624 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cf6612b7-1dd8-468a-9c6c-6825e76e4bcc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:28.106075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cf6612b7-1dd8-468a-9c6c-6825e76e4bcc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:28.106899 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cf6612b7-1dd8-468a-9c6c-6825e76e4bcc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:28.107367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cf6612b7-1dd8-468a-9c6c-6825e76e4bcc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:28.107793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cf6612b7-1dd8-468a-9c6c-6825e76e4bcc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:28.116247 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2f602120-6cc7-4249-ad14-c4c863d50de2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:28.116247 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2f602120-6cc7-4249-ad14-c4c863d50de2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:28.119999 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f602120-6cc7-4249-ad14-c4c863d50de2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:28.120508 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f602120-6cc7-4249-ad14-c4c863d50de2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:28.121464 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f602120-6cc7-4249-ad14-c4c863d50de2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:28.122224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f602120-6cc7-4249-ad14-c4c863d50de2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:28.122694 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f602120-6cc7-4249-ad14-c4c863d50de2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:28.123127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f602120-6cc7-4249-ad14-c4c863d50de2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:28.195145 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2f602120-6cc7-4249-ad14-c4c863d50de2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:28.203800 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2f602120-6cc7-4249-ad14-c4c863d50de2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.095439 Sep 08 14:04:28.204584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 997/1883] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:28 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:28.280028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-cf6612b7-1dd8-468a-9c6c-6825e76e4bcc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:28.290434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cf6612b7-1dd8-468a-9c6c-6825e76e4bcc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.201059 Sep 08 14:04:28.291084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 887/1884] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:28 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:28.470283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-11d35e46-8ed6-44ac-b798-bc18f5cbe485 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:28.472356 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-11d35e46-8ed6-44ac-b798-bc18f5cbe485 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:28.478873 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-11d35e46-8ed6-44ac-b798-bc18f5cbe485 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:28.479410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-11d35e46-8ed6-44ac-b798-bc18f5cbe485 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:28.479858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-11d35e46-8ed6-44ac-b798-bc18f5cbe485 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:28.481121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-11d35e46-8ed6-44ac-b798-bc18f5cbe485 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:28.481579 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-11d35e46-8ed6-44ac-b798-bc18f5cbe485 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:28.482032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-11d35e46-8ed6-44ac-b798-bc18f5cbe485 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:28.518434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-11d35e46-8ed6-44ac-b798-bc18f5cbe485 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052172 Sep 08 14:04:28.519007 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 998/1885] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:28 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:28.554519 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-56f29958-0377-4968-8ce0-e270f6626888 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:28.557990 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-56f29958-0377-4968-8ce0-e270f6626888 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:28.566653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-56f29958-0377-4968-8ce0-e270f6626888 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:28.567143 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-56f29958-0377-4968-8ce0-e270f6626888 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:28.567575 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-56f29958-0377-4968-8ce0-e270f6626888 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:28.568280 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-56f29958-0377-4968-8ce0-e270f6626888 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:28.568727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-56f29958-0377-4968-8ce0-e270f6626888 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:28.569146 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-56f29958-0377-4968-8ce0-e270f6626888 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:28.596678 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:28.602651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-56f29958-0377-4968-8ce0-e270f6626888 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053649 Sep 08 14:04:28.603246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 888/1886] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:28 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:28.922612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e719893a-7412-4fbf-853c-00c095ad66a9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:28.924727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e719893a-7412-4fbf-853c-00c095ad66a9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:28.931926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e719893a-7412-4fbf-853c-00c095ad66a9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:28.932418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e719893a-7412-4fbf-853c-00c095ad66a9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:28.932859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e719893a-7412-4fbf-853c-00c095ad66a9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:28.933573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e719893a-7412-4fbf-853c-00c095ad66a9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:28.934035 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e719893a-7412-4fbf-853c-00c095ad66a9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:28.934498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e719893a-7412-4fbf-853c-00c095ad66a9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:28.966112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e719893a-7412-4fbf-853c-00c095ad66a9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048271 Sep 08 14:04:28.966711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 999/1887] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:28 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:29.227505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-14ca3b52-857d-44ac-82d9-d9f469d53a8b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:29.228019 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-14ca3b52-857d-44ac-82d9-d9f469d53a8b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:29.234510 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-14ca3b52-857d-44ac-82d9-d9f469d53a8b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:29.234989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-14ca3b52-857d-44ac-82d9-d9f469d53a8b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:29.235417 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-14ca3b52-857d-44ac-82d9-d9f469d53a8b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:29.236109 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-14ca3b52-857d-44ac-82d9-d9f469d53a8b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:29.236553 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-14ca3b52-857d-44ac-82d9-d9f469d53a8b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:29.236969 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-14ca3b52-857d-44ac-82d9-d9f469d53a8b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:29.315713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-aa8b005e-e19e-41c8-b5f1-65fae36a9bb2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:29.315962 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-14ca3b52-857d-44ac-82d9-d9f469d53a8b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:29.316767 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-aa8b005e-e19e-41c8-b5f1-65fae36a9bb2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:29.327147 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa8b005e-e19e-41c8-b5f1-65fae36a9bb2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:29.327927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa8b005e-e19e-41c8-b5f1-65fae36a9bb2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:29.328599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa8b005e-e19e-41c8-b5f1-65fae36a9bb2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:29.329706 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa8b005e-e19e-41c8-b5f1-65fae36a9bb2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:29.330188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-14ca3b52-857d-44ac-82d9-d9f469d53a8b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.106182 Sep 08 14:04:29.330476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa8b005e-e19e-41c8-b5f1-65fae36a9bb2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:29.330935 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 889/1888] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:29 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:29.331151 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa8b005e-e19e-41c8-b5f1-65fae36a9bb2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:29.486394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-aa8b005e-e19e-41c8-b5f1-65fae36a9bb2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:29.494862 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-aa8b005e-e19e-41c8-b5f1-65fae36a9bb2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.184722 Sep 08 14:04:29.495950 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1000/1889] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:29 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:29.531598 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ab19feb6-312f-452b-9a89-362fece65f05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:29.534956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ab19feb6-312f-452b-9a89-362fece65f05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:29.544701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ab19feb6-312f-452b-9a89-362fece65f05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:29.545190 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ab19feb6-312f-452b-9a89-362fece65f05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:29.545632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ab19feb6-312f-452b-9a89-362fece65f05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:29.546376 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ab19feb6-312f-452b-9a89-362fece65f05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:29.546831 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ab19feb6-312f-452b-9a89-362fece65f05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:29.547251 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ab19feb6-312f-452b-9a89-362fece65f05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:29.581790 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ab19feb6-312f-452b-9a89-362fece65f05 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054289 Sep 08 14:04:29.582902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 890/1890] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:29 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:29.627611 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a1cf5f0f-52d3-468f-a436-24321e2d1d06 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:29.629807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a1cf5f0f-52d3-468f-a436-24321e2d1d06 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:29.638553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1cf5f0f-52d3-468f-a436-24321e2d1d06 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:29.639047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1cf5f0f-52d3-468f-a436-24321e2d1d06 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:29.639640 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1cf5f0f-52d3-468f-a436-24321e2d1d06 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:29.640537 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1cf5f0f-52d3-468f-a436-24321e2d1d06 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:29.640997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1cf5f0f-52d3-468f-a436-24321e2d1d06 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:29.641417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1cf5f0f-52d3-468f-a436-24321e2d1d06 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:29.674261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a1cf5f0f-52d3-468f-a436-24321e2d1d06 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051396 Sep 08 14:04:29.674835 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1001/1891] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:29 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:29.979304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c345c7ee-e32b-4530-a208-2d056ff5b962 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:29.982226 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c345c7ee-e32b-4530-a208-2d056ff5b962 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:29.992639 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c345c7ee-e32b-4530-a208-2d056ff5b962 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:29.993134 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c345c7ee-e32b-4530-a208-2d056ff5b962 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:29.993831 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c345c7ee-e32b-4530-a208-2d056ff5b962 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:29.994572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c345c7ee-e32b-4530-a208-2d056ff5b962 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:29.995024 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c345c7ee-e32b-4530-a208-2d056ff5b962 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:29.995479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c345c7ee-e32b-4530-a208-2d056ff5b962 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:30.039484 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c345c7ee-e32b-4530-a208-2d056ff5b962 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.064519 Sep 08 14:04:30.040064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 891/1892] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:29 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:04:30.358369 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fc93b2a9-5954-4347-bded-02c53f030593 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:30.358986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fc93b2a9-5954-4347-bded-02c53f030593 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:30.366110 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc93b2a9-5954-4347-bded-02c53f030593 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:30.366641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc93b2a9-5954-4347-bded-02c53f030593 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:30.367084 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc93b2a9-5954-4347-bded-02c53f030593 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:30.367797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc93b2a9-5954-4347-bded-02c53f030593 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:30.368513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc93b2a9-5954-4347-bded-02c53f030593 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:30.369021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fc93b2a9-5954-4347-bded-02c53f030593 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:30.437633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fc93b2a9-5954-4347-bded-02c53f030593 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:30.448166 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fc93b2a9-5954-4347-bded-02c53f030593 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.094607 Sep 08 14:04:30.449531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1002/1893] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:30 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:30.522761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4ac92623-7170-4136-a85e-31693dc3461d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:30.523495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4ac92623-7170-4136-a85e-31693dc3461d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:30.530921 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4ac92623-7170-4136-a85e-31693dc3461d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:30.531556 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4ac92623-7170-4136-a85e-31693dc3461d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:30.531914 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4ac92623-7170-4136-a85e-31693dc3461d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:30.532622 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4ac92623-7170-4136-a85e-31693dc3461d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:30.533075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4ac92623-7170-4136-a85e-31693dc3461d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:30.533631 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4ac92623-7170-4136-a85e-31693dc3461d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:30.597874 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-277074b3-8815-4985-949c-6bee5f36fe5d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:30.600072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-277074b3-8815-4985-949c-6bee5f36fe5d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:30.607256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-277074b3-8815-4985-949c-6bee5f36fe5d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:30.607739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-277074b3-8815-4985-949c-6bee5f36fe5d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:30.608172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-277074b3-8815-4985-949c-6bee5f36fe5d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:30.608924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-277074b3-8815-4985-949c-6bee5f36fe5d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:30.609319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-277074b3-8815-4985-949c-6bee5f36fe5d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:30.609736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-277074b3-8815-4985-949c-6bee5f36fe5d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:30.646983 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-277074b3-8815-4985-949c-6bee5f36fe5d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049591 Sep 08 14:04:30.646983 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1003/1894] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:30 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:30.673439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4ac92623-7170-4136-a85e-31693dc3461d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:30.682350 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4ac92623-7170-4136-a85e-31693dc3461d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.164978 Sep 08 14:04:30.682973 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 892/1895] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:30 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:30.689097 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f5d95e97-6cd5-436f-80c7-b620225740f1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:30.691409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f5d95e97-6cd5-436f-80c7-b620225740f1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:30.699013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f5d95e97-6cd5-436f-80c7-b620225740f1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:30.699518 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f5d95e97-6cd5-436f-80c7-b620225740f1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:30.700023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f5d95e97-6cd5-436f-80c7-b620225740f1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:30.700693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f5d95e97-6cd5-436f-80c7-b620225740f1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:30.701258 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f5d95e97-6cd5-436f-80c7-b620225740f1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:30.701697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f5d95e97-6cd5-436f-80c7-b620225740f1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:30.740749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f5d95e97-6cd5-436f-80c7-b620225740f1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055212 Sep 08 14:04:30.741712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1004/1896] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:30 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:31.056712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0cb62b04-1625-4ccc-b028-181e9dbe433d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:31.060869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0cb62b04-1625-4ccc-b028-181e9dbe433d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:31.069760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0cb62b04-1625-4ccc-b028-181e9dbe433d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:31.070534 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0cb62b04-1625-4ccc-b028-181e9dbe433d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:31.071202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0cb62b04-1625-4ccc-b028-181e9dbe433d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:31.072278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0cb62b04-1625-4ccc-b028-181e9dbe433d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:31.072970 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0cb62b04-1625-4ccc-b028-181e9dbe433d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:31.073712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0cb62b04-1625-4ccc-b028-181e9dbe433d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:31.117162 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0cb62b04-1625-4ccc-b028-181e9dbe433d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.067113 Sep 08 14:04:31.118513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 893/1897] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:31 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:31.474655 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-283534b0-1e57-433d-bbef-4790c0160743 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:31.475260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-283534b0-1e57-433d-bbef-4790c0160743 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:31.482661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-283534b0-1e57-433d-bbef-4790c0160743 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:31.483167 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-283534b0-1e57-433d-bbef-4790c0160743 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:31.483595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-283534b0-1e57-433d-bbef-4790c0160743 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:31.484285 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-283534b0-1e57-433d-bbef-4790c0160743 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:31.484823 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-283534b0-1e57-433d-bbef-4790c0160743 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:31.485531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-283534b0-1e57-433d-bbef-4790c0160743 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:31.552992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-283534b0-1e57-433d-bbef-4790c0160743 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:31.569440 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-283534b0-1e57-433d-bbef-4790c0160743 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.098600 Sep 08 14:04:31.570431 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1005/1898] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:31 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:31.658658 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-418a2b62-a23f-4bd9-960e-7d630d067ae3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:31.661196 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-418a2b62-a23f-4bd9-960e-7d630d067ae3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:31.668316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-418a2b62-a23f-4bd9-960e-7d630d067ae3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:31.668827 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-418a2b62-a23f-4bd9-960e-7d630d067ae3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:31.669261 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-418a2b62-a23f-4bd9-960e-7d630d067ae3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:31.669949 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-418a2b62-a23f-4bd9-960e-7d630d067ae3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:31.670424 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-418a2b62-a23f-4bd9-960e-7d630d067ae3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:31.670844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-418a2b62-a23f-4bd9-960e-7d630d067ae3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:31.702527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-418a2b62-a23f-4bd9-960e-7d630d067ae3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050318 Sep 08 14:04:31.704628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 894/1899] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:31 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:31.715960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6308b454-e9c6-4141-b76b-0c4faa4b769a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:31.716722 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6308b454-e9c6-4141-b76b-0c4faa4b769a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:31.723451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6308b454-e9c6-4141-b76b-0c4faa4b769a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:31.723977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6308b454-e9c6-4141-b76b-0c4faa4b769a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:31.724414 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6308b454-e9c6-4141-b76b-0c4faa4b769a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:31.725186 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6308b454-e9c6-4141-b76b-0c4faa4b769a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:31.725591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6308b454-e9c6-4141-b76b-0c4faa4b769a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:31.726040 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6308b454-e9c6-4141-b76b-0c4faa4b769a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:31.757934 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7dec7324-311c-4752-a2f9-dd2a70cd5905 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:31.761455 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7dec7324-311c-4752-a2f9-dd2a70cd5905 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:31.771383 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7dec7324-311c-4752-a2f9-dd2a70cd5905 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:31.771974 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7dec7324-311c-4752-a2f9-dd2a70cd5905 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:31.772585 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7dec7324-311c-4752-a2f9-dd2a70cd5905 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:31.773619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7dec7324-311c-4752-a2f9-dd2a70cd5905 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:31.774227 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7dec7324-311c-4752-a2f9-dd2a70cd5905 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:31.775061 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7dec7324-311c-4752-a2f9-dd2a70cd5905 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:31.836591 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7dec7324-311c-4752-a2f9-dd2a70cd5905 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.085304 Sep 08 14:04:31.837189 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 895/1900] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:31 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:31.906352 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6308b454-e9c6-4141-b76b-0c4faa4b769a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:31.915870 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6308b454-e9c6-4141-b76b-0c4faa4b769a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.205801 Sep 08 14:04:31.916963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1006/1901] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:31 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:32.131411 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-820bf458-cc95-4498-bec2-9faa9f225f83 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:32.133915 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-820bf458-cc95-4498-bec2-9faa9f225f83 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:32.141402 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-820bf458-cc95-4498-bec2-9faa9f225f83 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:32.141873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-820bf458-cc95-4498-bec2-9faa9f225f83 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:32.142343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-820bf458-cc95-4498-bec2-9faa9f225f83 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:32.143111 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-820bf458-cc95-4498-bec2-9faa9f225f83 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:32.143646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-820bf458-cc95-4498-bec2-9faa9f225f83 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:32.144071 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-820bf458-cc95-4498-bec2-9faa9f225f83 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:32.182972 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-820bf458-cc95-4498-bec2-9faa9f225f83 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055088 Sep 08 14:04:32.183953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 896/1902] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:32 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:32.605489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6f96ef0a-415d-48b5-947a-917554f3609e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:32.606122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6f96ef0a-415d-48b5-947a-917554f3609e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:32.618529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f96ef0a-415d-48b5-947a-917554f3609e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:32.619323 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f96ef0a-415d-48b5-947a-917554f3609e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:32.620097 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f96ef0a-415d-48b5-947a-917554f3609e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:32.621262 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f96ef0a-415d-48b5-947a-917554f3609e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:32.622034 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f96ef0a-415d-48b5-947a-917554f3609e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:32.622901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f96ef0a-415d-48b5-947a-917554f3609e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:32.714898 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6f96ef0a-415d-48b5-947a-917554f3609e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:32.720137 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-19f8ec15-0657-44fb-9f8b-79bdb0551ffb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:32.722363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-19f8ec15-0657-44fb-9f8b-79bdb0551ffb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:32.723271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6f96ef0a-415d-48b5-947a-917554f3609e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.123177 Sep 08 14:04:32.723892 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1007/1903] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:32 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:32.729175 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19f8ec15-0657-44fb-9f8b-79bdb0551ffb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:32.730347 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19f8ec15-0657-44fb-9f8b-79bdb0551ffb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:32.730840 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19f8ec15-0657-44fb-9f8b-79bdb0551ffb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:32.731551 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19f8ec15-0657-44fb-9f8b-79bdb0551ffb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:32.732008 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19f8ec15-0657-44fb-9f8b-79bdb0551ffb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:32.732430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19f8ec15-0657-44fb-9f8b-79bdb0551ffb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:32.765533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-19f8ec15-0657-44fb-9f8b-79bdb0551ffb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053521 Sep 08 14:04:32.766110 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 897/1904] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:32 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:32.849533 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8068d9f8-6e41-494c-8c8f-148e0f78c3bb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:32.851729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8068d9f8-6e41-494c-8c8f-148e0f78c3bb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:32.859319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8068d9f8-6e41-494c-8c8f-148e0f78c3bb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:32.859739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8068d9f8-6e41-494c-8c8f-148e0f78c3bb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:32.860176 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8068d9f8-6e41-494c-8c8f-148e0f78c3bb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:32.860875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8068d9f8-6e41-494c-8c8f-148e0f78c3bb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:32.861331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8068d9f8-6e41-494c-8c8f-148e0f78c3bb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:32.861807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8068d9f8-6e41-494c-8c8f-148e0f78c3bb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:32.905013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8068d9f8-6e41-494c-8c8f-148e0f78c3bb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059556 Sep 08 14:04:32.905683 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1008/1905] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:32 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:32.949114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-18314b0f-6f88-4757-ba7c-e0dde45d9843 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:32.949753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-18314b0f-6f88-4757-ba7c-e0dde45d9843 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:32.957870 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18314b0f-6f88-4757-ba7c-e0dde45d9843 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:32.958724 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18314b0f-6f88-4757-ba7c-e0dde45d9843 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:32.959249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18314b0f-6f88-4757-ba7c-e0dde45d9843 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:32.960160 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18314b0f-6f88-4757-ba7c-e0dde45d9843 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:32.960767 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18314b0f-6f88-4757-ba7c-e0dde45d9843 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:32.961239 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18314b0f-6f88-4757-ba7c-e0dde45d9843 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:33.121449 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-18314b0f-6f88-4757-ba7c-e0dde45d9843 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:33.129085 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-18314b0f-6f88-4757-ba7c-e0dde45d9843 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.182917 Sep 08 14:04:33.129085 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 898/1906] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:32 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:33.199248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d6ad3729-5993-43cc-8cd6-36559692aa7a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:33.201612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d6ad3729-5993-43cc-8cd6-36559692aa7a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:33.208835 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d6ad3729-5993-43cc-8cd6-36559692aa7a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:33.210340 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d6ad3729-5993-43cc-8cd6-36559692aa7a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:33.210340 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d6ad3729-5993-43cc-8cd6-36559692aa7a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:33.210732 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d6ad3729-5993-43cc-8cd6-36559692aa7a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:33.210944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d6ad3729-5993-43cc-8cd6-36559692aa7a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:33.211370 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d6ad3729-5993-43cc-8cd6-36559692aa7a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:33.228355 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:33.252172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d6ad3729-5993-43cc-8cd6-36559692aa7a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057470 Sep 08 14:04:33.254568 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1009/1907] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:33 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:04:33.749578 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-12fc0c7e-034c-4a8c-9833-d2a6d9f88ef9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:33.751201 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-12fc0c7e-034c-4a8c-9833-d2a6d9f88ef9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:33.757062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-12fc0c7e-034c-4a8c-9833-d2a6d9f88ef9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:33.757528 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-12fc0c7e-034c-4a8c-9833-d2a6d9f88ef9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:33.757960 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-12fc0c7e-034c-4a8c-9833-d2a6d9f88ef9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:33.758736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-12fc0c7e-034c-4a8c-9833-d2a6d9f88ef9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:33.759392 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-12fc0c7e-034c-4a8c-9833-d2a6d9f88ef9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:33.759708 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-12fc0c7e-034c-4a8c-9833-d2a6d9f88ef9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:33.779387 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c0d6d84b-211a-4f2a-998a-6d65981cc0ad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:33.781614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c0d6d84b-211a-4f2a-998a-6d65981cc0ad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:33.789615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0d6d84b-211a-4f2a-998a-6d65981cc0ad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:33.790102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0d6d84b-211a-4f2a-998a-6d65981cc0ad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:33.790573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0d6d84b-211a-4f2a-998a-6d65981cc0ad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:33.791527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0d6d84b-211a-4f2a-998a-6d65981cc0ad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:33.792176 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0d6d84b-211a-4f2a-998a-6d65981cc0ad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:33.792608 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0d6d84b-211a-4f2a-998a-6d65981cc0ad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:33.832828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c0d6d84b-211a-4f2a-998a-6d65981cc0ad tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058442 Sep 08 14:04:33.833937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1010/1908] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:33 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:33.837353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-12fc0c7e-034c-4a8c-9833-d2a6d9f88ef9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:33.847149 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-12fc0c7e-034c-4a8c-9833-d2a6d9f88ef9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.101619 Sep 08 14:04:33.847781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 899/1909] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:33 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:33.919006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-57edad0f-6067-4d16-b6c4-45e6828d9593 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:33.921189 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-57edad0f-6067-4d16-b6c4-45e6828d9593 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:33.927498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57edad0f-6067-4d16-b6c4-45e6828d9593 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:33.927968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57edad0f-6067-4d16-b6c4-45e6828d9593 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:33.928402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57edad0f-6067-4d16-b6c4-45e6828d9593 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:33.929101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57edad0f-6067-4d16-b6c4-45e6828d9593 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:33.929574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57edad0f-6067-4d16-b6c4-45e6828d9593 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:33.929993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57edad0f-6067-4d16-b6c4-45e6828d9593 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:33.969794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-57edad0f-6067-4d16-b6c4-45e6828d9593 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055598 Sep 08 14:04:33.970465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1011/1910] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:33 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:34.160115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c8c3bc84-1118-4ead-8e4e-40237fa97ee8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:34.160722 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c8c3bc84-1118-4ead-8e4e-40237fa97ee8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:34.167493 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c8c3bc84-1118-4ead-8e4e-40237fa97ee8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:34.167986 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c8c3bc84-1118-4ead-8e4e-40237fa97ee8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:34.168420 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c8c3bc84-1118-4ead-8e4e-40237fa97ee8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:34.169199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c8c3bc84-1118-4ead-8e4e-40237fa97ee8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:34.169590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c8c3bc84-1118-4ead-8e4e-40237fa97ee8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:34.170016 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c8c3bc84-1118-4ead-8e4e-40237fa97ee8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:34.266650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-93e08af2-494e-4fe5-ac1a-b6e45078c9aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:34.268903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-93e08af2-494e-4fe5-ac1a-b6e45078c9aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:34.275378 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93e08af2-494e-4fe5-ac1a-b6e45078c9aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:34.275877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93e08af2-494e-4fe5-ac1a-b6e45078c9aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:34.276688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93e08af2-494e-4fe5-ac1a-b6e45078c9aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:34.277412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93e08af2-494e-4fe5-ac1a-b6e45078c9aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:34.277869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93e08af2-494e-4fe5-ac1a-b6e45078c9aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:34.278412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-93e08af2-494e-4fe5-ac1a-b6e45078c9aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:34.312315 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-93e08af2-494e-4fe5-ac1a-b6e45078c9aa tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049892 Sep 08 14:04:34.312937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1012/1911] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:34 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:34.320197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c8c3bc84-1118-4ead-8e4e-40237fa97ee8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:34.334876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c8c3bc84-1118-4ead-8e4e-40237fa97ee8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.172637 Sep 08 14:04:34.334876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 900/1912] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:34 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:34.851206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-519edacf-09f1-45f6-8a4c-229e917a85bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:34.853397 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-519edacf-09f1-45f6-8a4c-229e917a85bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:34.874057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-519edacf-09f1-45f6-8a4c-229e917a85bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:34.874920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-519edacf-09f1-45f6-8a4c-229e917a85bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:34.878458 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-68496d02-58bd-40c2-8991-69650a87f2c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:34.879067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-519edacf-09f1-45f6-8a4c-229e917a85bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:34.879475 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-68496d02-58bd-40c2-8991-69650a87f2c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:34.879612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-519edacf-09f1-45f6-8a4c-229e917a85bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:34.880001 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-519edacf-09f1-45f6-8a4c-229e917a85bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:34.880435 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-519edacf-09f1-45f6-8a4c-229e917a85bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:34.888897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-68496d02-58bd-40c2-8991-69650a87f2c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:34.889800 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-68496d02-58bd-40c2-8991-69650a87f2c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:34.890550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-68496d02-58bd-40c2-8991-69650a87f2c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:34.891908 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-68496d02-58bd-40c2-8991-69650a87f2c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:34.892877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-68496d02-58bd-40c2-8991-69650a87f2c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:34.893774 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-68496d02-58bd-40c2-8991-69650a87f2c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:34.927942 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-519edacf-09f1-45f6-8a4c-229e917a85bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.084118 Sep 08 14:04:34.929186 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1013/1913] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:34 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:34.983952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-68496d02-58bd-40c2-8991-69650a87f2c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:34.986690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-984e07ad-8de0-4db6-97bd-4c5bf15b2ed7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:34.989951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-984e07ad-8de0-4db6-97bd-4c5bf15b2ed7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:34.996849 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-68496d02-58bd-40c2-8991-69650a87f2c6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.125127 Sep 08 14:04:34.997775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 901/1914] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:34 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:34.999599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-984e07ad-8de0-4db6-97bd-4c5bf15b2ed7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:35.000438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-984e07ad-8de0-4db6-97bd-4c5bf15b2ed7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:35.000995 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-984e07ad-8de0-4db6-97bd-4c5bf15b2ed7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:35.002193 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-984e07ad-8de0-4db6-97bd-4c5bf15b2ed7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:35.002970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-984e07ad-8de0-4db6-97bd-4c5bf15b2ed7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:35.003706 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-984e07ad-8de0-4db6-97bd-4c5bf15b2ed7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:35.048091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-984e07ad-8de0-4db6-97bd-4c5bf15b2ed7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.068294 Sep 08 14:04:35.048653 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1014/1915] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:34 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:35.327728 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a6a5abb2-9b58-4aa8-a3e1-ce7b1f1b9e36 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:35.331091 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a6a5abb2-9b58-4aa8-a3e1-ce7b1f1b9e36 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:35.341297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6a5abb2-9b58-4aa8-a3e1-ce7b1f1b9e36 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:35.342157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6a5abb2-9b58-4aa8-a3e1-ce7b1f1b9e36 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:35.342834 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6a5abb2-9b58-4aa8-a3e1-ce7b1f1b9e36 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:35.343964 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6a5abb2-9b58-4aa8-a3e1-ce7b1f1b9e36 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:35.344704 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6a5abb2-9b58-4aa8-a3e1-ce7b1f1b9e36 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:35.345396 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6a5abb2-9b58-4aa8-a3e1-ce7b1f1b9e36 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:35.361254 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ecf01583-047a-457e-b529-270b558d8d41 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:35.361905 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ecf01583-047a-457e-b529-270b558d8d41 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:35.369893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ecf01583-047a-457e-b529-270b558d8d41 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:35.370423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ecf01583-047a-457e-b529-270b558d8d41 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:35.370860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ecf01583-047a-457e-b529-270b558d8d41 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:35.371560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ecf01583-047a-457e-b529-270b558d8d41 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:35.372033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ecf01583-047a-457e-b529-270b558d8d41 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:35.372454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ecf01583-047a-457e-b529-270b558d8d41 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:35.391602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a6a5abb2-9b58-4aa8-a3e1-ce7b1f1b9e36 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.070122 Sep 08 14:04:35.392532 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 902/1916] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:35 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:35.557867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ecf01583-047a-457e-b529-270b558d8d41 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:35.567143 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ecf01583-047a-457e-b529-270b558d8d41 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.209896 Sep 08 14:04:35.567723 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1015/1917] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:35 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:35.941732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-792f6475-078e-409a-9b3a-96e043924a0a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:35.943836 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-792f6475-078e-409a-9b3a-96e043924a0a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:35.951010 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-792f6475-078e-409a-9b3a-96e043924a0a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:35.951591 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-792f6475-078e-409a-9b3a-96e043924a0a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:35.952713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-792f6475-078e-409a-9b3a-96e043924a0a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:35.952713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-792f6475-078e-409a-9b3a-96e043924a0a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:35.953223 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-792f6475-078e-409a-9b3a-96e043924a0a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:35.953820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-792f6475-078e-409a-9b3a-96e043924a0a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:35.986074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-792f6475-078e-409a-9b3a-96e043924a0a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048443 Sep 08 14:04:35.987468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 903/1918] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:35 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:36.021151 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d66040bd-8799-4a90-ae60-84ca42f77bb9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:36.021752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d66040bd-8799-4a90-ae60-84ca42f77bb9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:36.029189 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d66040bd-8799-4a90-ae60-84ca42f77bb9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:36.029680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d66040bd-8799-4a90-ae60-84ca42f77bb9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:36.030118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d66040bd-8799-4a90-ae60-84ca42f77bb9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:36.030832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d66040bd-8799-4a90-ae60-84ca42f77bb9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:36.031285 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d66040bd-8799-4a90-ae60-84ca42f77bb9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:36.031708 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d66040bd-8799-4a90-ae60-84ca42f77bb9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:36.062795 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-913a18fd-c8dc-4e41-8f57-ab132576dadb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:36.071874 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-913a18fd-c8dc-4e41-8f57-ab132576dadb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:36.081593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-913a18fd-c8dc-4e41-8f57-ab132576dadb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:36.092628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-913a18fd-c8dc-4e41-8f57-ab132576dadb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:36.092628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-913a18fd-c8dc-4e41-8f57-ab132576dadb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:36.092628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-913a18fd-c8dc-4e41-8f57-ab132576dadb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:36.092628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-913a18fd-c8dc-4e41-8f57-ab132576dadb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:36.093049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-913a18fd-c8dc-4e41-8f57-ab132576dadb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:36.101809 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d66040bd-8799-4a90-ae60-84ca42f77bb9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:36.110374 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d66040bd-8799-4a90-ae60-84ca42f77bb9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.093156 Sep 08 14:04:36.110947 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1016/1919] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:36 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:36.127292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-913a18fd-c8dc-4e41-8f57-ab132576dadb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.068583 Sep 08 14:04:36.127857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 904/1920] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:36 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:36.407290 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6798145c-ca6d-4a44-b5a3-438a22039e11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:36.409498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6798145c-ca6d-4a44-b5a3-438a22039e11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:36.418247 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6798145c-ca6d-4a44-b5a3-438a22039e11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:36.419045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6798145c-ca6d-4a44-b5a3-438a22039e11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:36.419766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6798145c-ca6d-4a44-b5a3-438a22039e11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:36.420906 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6798145c-ca6d-4a44-b5a3-438a22039e11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:36.421688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6798145c-ca6d-4a44-b5a3-438a22039e11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:36.422422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6798145c-ca6d-4a44-b5a3-438a22039e11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:36.462122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6798145c-ca6d-4a44-b5a3-438a22039e11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058477 Sep 08 14:04:36.463484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1017/1921] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:36 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:04:36.591795 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fe5dedd0-4d74-4acd-b561-2cfc15bf47f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:36.592467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fe5dedd0-4d74-4acd-b561-2cfc15bf47f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:36.600675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fe5dedd0-4d74-4acd-b561-2cfc15bf47f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:36.601513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fe5dedd0-4d74-4acd-b561-2cfc15bf47f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:36.602242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fe5dedd0-4d74-4acd-b561-2cfc15bf47f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:36.603360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fe5dedd0-4d74-4acd-b561-2cfc15bf47f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:36.604099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fe5dedd0-4d74-4acd-b561-2cfc15bf47f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:36.604761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fe5dedd0-4d74-4acd-b561-2cfc15bf47f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:36.638115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:36.774374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-fe5dedd0-4d74-4acd-b561-2cfc15bf47f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:36.782879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fe5dedd0-4d74-4acd-b561-2cfc15bf47f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.195152 Sep 08 14:04:36.783458 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 905/1922] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:36 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:37.003482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-77f869b8-e475-4a8a-a7d4-be4d4d25c718 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:37.005979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-77f869b8-e475-4a8a-a7d4-be4d4d25c718 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:37.012863 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77f869b8-e475-4a8a-a7d4-be4d4d25c718 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:37.013355 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77f869b8-e475-4a8a-a7d4-be4d4d25c718 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:37.013797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77f869b8-e475-4a8a-a7d4-be4d4d25c718 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:37.014533 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77f869b8-e475-4a8a-a7d4-be4d4d25c718 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:37.014991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77f869b8-e475-4a8a-a7d4-be4d4d25c718 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:37.015420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77f869b8-e475-4a8a-a7d4-be4d4d25c718 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:37.047505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-77f869b8-e475-4a8a-a7d4-be4d4d25c718 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051008 Sep 08 14:04:37.048255 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1018/1923] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:36 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:37.135136 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b81761ad-6c54-4a25-8a83-eb5cdced29c0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:37.135944 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b81761ad-6c54-4a25-8a83-eb5cdced29c0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:37.140952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2c91dd88-f637-4c61-a28e-40024013c482 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:37.142851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b81761ad-6c54-4a25-8a83-eb5cdced29c0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:37.143343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b81761ad-6c54-4a25-8a83-eb5cdced29c0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:37.143470 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2c91dd88-f637-4c61-a28e-40024013c482 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:37.143782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b81761ad-6c54-4a25-8a83-eb5cdced29c0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:37.144486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b81761ad-6c54-4a25-8a83-eb5cdced29c0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:37.144945 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b81761ad-6c54-4a25-8a83-eb5cdced29c0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:37.145373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b81761ad-6c54-4a25-8a83-eb5cdced29c0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:37.149501 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c91dd88-f637-4c61-a28e-40024013c482 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:37.150025 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c91dd88-f637-4c61-a28e-40024013c482 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:37.150661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c91dd88-f637-4c61-a28e-40024013c482 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:37.151895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c91dd88-f637-4c61-a28e-40024013c482 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:37.152354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c91dd88-f637-4c61-a28e-40024013c482 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:37.152836 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c91dd88-f637-4c61-a28e-40024013c482 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:37.186565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2c91dd88-f637-4c61-a28e-40024013c482 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050170 Sep 08 14:04:37.187142 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1019/1924] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:37 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:37.211195 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b81761ad-6c54-4a25-8a83-eb5cdced29c0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:37.220835 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b81761ad-6c54-4a25-8a83-eb5cdced29c0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.090264 Sep 08 14:04:37.221426 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 906/1925] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:37 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:37.478531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e236f0fa-6ae1-4818-9879-2b4a1805b4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:37.480745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e236f0fa-6ae1-4818-9879-2b4a1805b4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:37.487288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e236f0fa-6ae1-4818-9879-2b4a1805b4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:37.487788 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e236f0fa-6ae1-4818-9879-2b4a1805b4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:37.488226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e236f0fa-6ae1-4818-9879-2b4a1805b4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:37.488931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e236f0fa-6ae1-4818-9879-2b4a1805b4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:37.489400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e236f0fa-6ae1-4818-9879-2b4a1805b4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:37.489819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e236f0fa-6ae1-4818-9879-2b4a1805b4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:37.524798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e236f0fa-6ae1-4818-9879-2b4a1805b4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050386 Sep 08 14:04:37.525380 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1020/1926] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:37 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:37.808672 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5543389b-0af6-43a1-ad2e-573e6fece8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:37.809374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5543389b-0af6-43a1-ad2e-573e6fece8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:37.817379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5543389b-0af6-43a1-ad2e-573e6fece8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:37.818343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5543389b-0af6-43a1-ad2e-573e6fece8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:37.819059 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5543389b-0af6-43a1-ad2e-573e6fece8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:37.820223 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5543389b-0af6-43a1-ad2e-573e6fece8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:37.820974 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5543389b-0af6-43a1-ad2e-573e6fece8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:37.821690 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5543389b-0af6-43a1-ad2e-573e6fece8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:37.978188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5543389b-0af6-43a1-ad2e-573e6fece8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:37.988167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5543389b-0af6-43a1-ad2e-573e6fece8d9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.185400 Sep 08 14:04:37.989085 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 907/1927] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:37 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:38.063009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-668c4035-0b0d-4f01-92fb-c7b8061f459e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:38.065359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-668c4035-0b0d-4f01-92fb-c7b8061f459e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:38.072492 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-668c4035-0b0d-4f01-92fb-c7b8061f459e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:38.073607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-668c4035-0b0d-4f01-92fb-c7b8061f459e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:38.074046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-668c4035-0b0d-4f01-92fb-c7b8061f459e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:38.074773 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-668c4035-0b0d-4f01-92fb-c7b8061f459e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:38.075222 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-668c4035-0b0d-4f01-92fb-c7b8061f459e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:38.075644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-668c4035-0b0d-4f01-92fb-c7b8061f459e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:38.126425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-668c4035-0b0d-4f01-92fb-c7b8061f459e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069821 Sep 08 14:04:38.126996 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1021/1928] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:38 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:38.201425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-289a07ac-7363-4ea4-be50-5a7636fd641c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:38.203653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-289a07ac-7363-4ea4-be50-5a7636fd641c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:38.210667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-289a07ac-7363-4ea4-be50-5a7636fd641c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:38.211242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-289a07ac-7363-4ea4-be50-5a7636fd641c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:38.211653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-289a07ac-7363-4ea4-be50-5a7636fd641c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:38.212352 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-289a07ac-7363-4ea4-be50-5a7636fd641c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:38.212818 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-289a07ac-7363-4ea4-be50-5a7636fd641c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:38.213560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-289a07ac-7363-4ea4-be50-5a7636fd641c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:38.246072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-aff2f666-5c20-42cd-8cdd-2eaee9499385 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:38.246442 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-289a07ac-7363-4ea4-be50-5a7636fd641c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049063 Sep 08 14:04:38.247092 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 908/1929] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:38 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:38.247325 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-aff2f666-5c20-42cd-8cdd-2eaee9499385 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:38.253749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aff2f666-5c20-42cd-8cdd-2eaee9499385 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:38.254277 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aff2f666-5c20-42cd-8cdd-2eaee9499385 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:38.254721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aff2f666-5c20-42cd-8cdd-2eaee9499385 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:38.255464 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aff2f666-5c20-42cd-8cdd-2eaee9499385 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:38.255953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aff2f666-5c20-42cd-8cdd-2eaee9499385 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:38.256415 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aff2f666-5c20-42cd-8cdd-2eaee9499385 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:38.323439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-aff2f666-5c20-42cd-8cdd-2eaee9499385 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:38.331464 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-aff2f666-5c20-42cd-8cdd-2eaee9499385 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.090399 Sep 08 14:04:38.332022 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1022/1930] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:38 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:38.538228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5d2771bb-bb15-4d45-abf8-0da3ef3bfa6d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:38.540391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5d2771bb-bb15-4d45-abf8-0da3ef3bfa6d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:38.548968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d2771bb-bb15-4d45-abf8-0da3ef3bfa6d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:38.549538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d2771bb-bb15-4d45-abf8-0da3ef3bfa6d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:38.549986 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d2771bb-bb15-4d45-abf8-0da3ef3bfa6d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:38.550733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d2771bb-bb15-4d45-abf8-0da3ef3bfa6d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:38.551181 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d2771bb-bb15-4d45-abf8-0da3ef3bfa6d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:38.551602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d2771bb-bb15-4d45-abf8-0da3ef3bfa6d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:38.599751 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5d2771bb-bb15-4d45-abf8-0da3ef3bfa6d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.065485 Sep 08 14:04:38.600512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 909/1931] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:38 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:39.018354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9ab8fd97-f6d4-4468-b8e0-0a23967ad2d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:39.018851 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9ab8fd97-f6d4-4468-b8e0-0a23967ad2d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:39.025000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9ab8fd97-f6d4-4468-b8e0-0a23967ad2d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:39.025485 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9ab8fd97-f6d4-4468-b8e0-0a23967ad2d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:39.026711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9ab8fd97-f6d4-4468-b8e0-0a23967ad2d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:39.034653 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9ab8fd97-f6d4-4468-b8e0-0a23967ad2d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:39.034653 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9ab8fd97-f6d4-4468-b8e0-0a23967ad2d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:39.034653 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9ab8fd97-f6d4-4468-b8e0-0a23967ad2d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:39.140574 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-26e588c6-6cba-4ea8-b373-33950ebcf552 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:39.144393 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-26e588c6-6cba-4ea8-b373-33950ebcf552 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:39.152256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26e588c6-6cba-4ea8-b373-33950ebcf552 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:39.152751 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26e588c6-6cba-4ea8-b373-33950ebcf552 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:39.153183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26e588c6-6cba-4ea8-b373-33950ebcf552 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:39.153890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26e588c6-6cba-4ea8-b373-33950ebcf552 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:39.154380 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26e588c6-6cba-4ea8-b373-33950ebcf552 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:39.154793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26e588c6-6cba-4ea8-b373-33950ebcf552 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:39.188745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9ab8fd97-f6d4-4468-b8e0-0a23967ad2d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:39.191537 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-26e588c6-6cba-4ea8-b373-33950ebcf552 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055130 Sep 08 14:04:39.198222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 910/1932] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:39 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:39.198583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9ab8fd97-f6d4-4468-b8e0-0a23967ad2d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.182585 Sep 08 14:04:39.198583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1023/1933] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:39 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:39.261109 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b3295371-97de-4b14-930f-a124f1761fb8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:39.263331 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b3295371-97de-4b14-930f-a124f1761fb8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:39.271977 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3295371-97de-4b14-930f-a124f1761fb8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:39.272382 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3295371-97de-4b14-930f-a124f1761fb8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:39.272870 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3295371-97de-4b14-930f-a124f1761fb8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:39.273580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3295371-97de-4b14-930f-a124f1761fb8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:39.274024 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3295371-97de-4b14-930f-a124f1761fb8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:39.274477 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3295371-97de-4b14-930f-a124f1761fb8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:39.306922 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b3295371-97de-4b14-930f-a124f1761fb8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051597 Sep 08 14:04:39.308082 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 911/1934] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:39 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:39.354673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b262abaf-adf7-4bf5-a13d-20ac2060b688 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:39.355292 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b262abaf-adf7-4bf5-a13d-20ac2060b688 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:39.361583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b262abaf-adf7-4bf5-a13d-20ac2060b688 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:39.362079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b262abaf-adf7-4bf5-a13d-20ac2060b688 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:39.362548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b262abaf-adf7-4bf5-a13d-20ac2060b688 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:39.363261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b262abaf-adf7-4bf5-a13d-20ac2060b688 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:39.363740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b262abaf-adf7-4bf5-a13d-20ac2060b688 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:39.364145 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b262abaf-adf7-4bf5-a13d-20ac2060b688 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:39.428916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b262abaf-adf7-4bf5-a13d-20ac2060b688 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:39.439733 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b262abaf-adf7-4bf5-a13d-20ac2060b688 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.088940 Sep 08 14:04:39.440317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1024/1935] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:39 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:39.613221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-73d9635b-b024-4daf-bf1e-80a7f8f8a4e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:39.615456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-73d9635b-b024-4daf-bf1e-80a7f8f8a4e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:39.622673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73d9635b-b024-4daf-bf1e-80a7f8f8a4e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:39.623162 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73d9635b-b024-4daf-bf1e-80a7f8f8a4e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:39.623593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73d9635b-b024-4daf-bf1e-80a7f8f8a4e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:39.624297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73d9635b-b024-4daf-bf1e-80a7f8f8a4e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:39.624744 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73d9635b-b024-4daf-bf1e-80a7f8f8a4e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:39.625165 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73d9635b-b024-4daf-bf1e-80a7f8f8a4e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:39.658330 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-73d9635b-b024-4daf-bf1e-80a7f8f8a4e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049106 Sep 08 14:04:39.658912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 912/1936] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:39 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:40.204478 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ccf35af4-e61c-41b9-9f8d-0d68684183f4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:40.206759 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ccf35af4-e61c-41b9-9f8d-0d68684183f4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:40.214078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ccf35af4-e61c-41b9-9f8d-0d68684183f4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:40.214879 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ccf35af4-e61c-41b9-9f8d-0d68684183f4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:40.215331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ccf35af4-e61c-41b9-9f8d-0d68684183f4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:40.216099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ccf35af4-e61c-41b9-9f8d-0d68684183f4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:40.216873 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ccf35af4-e61c-41b9-9f8d-0d68684183f4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:40.217496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ccf35af4-e61c-41b9-9f8d-0d68684183f4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:40.223511 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-48bd4652-8c21-454b-87e5-ee57c8a3f7f1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:40.224420 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-48bd4652-8c21-454b-87e5-ee57c8a3f7f1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:40.231824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-48bd4652-8c21-454b-87e5-ee57c8a3f7f1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:40.232311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-48bd4652-8c21-454b-87e5-ee57c8a3f7f1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:40.232748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-48bd4652-8c21-454b-87e5-ee57c8a3f7f1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:40.233451 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-48bd4652-8c21-454b-87e5-ee57c8a3f7f1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:40.233903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-48bd4652-8c21-454b-87e5-ee57c8a3f7f1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:40.234450 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-48bd4652-8c21-454b-87e5-ee57c8a3f7f1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:40.254171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ccf35af4-e61c-41b9-9f8d-0d68684183f4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053615 Sep 08 14:04:40.254789 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1025/1937] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:40 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:40.321328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-824d73e5-e32e-4baf-8f4e-8ea21b7bd782 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:40.323786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-824d73e5-e32e-4baf-8f4e-8ea21b7bd782 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:40.331682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-824d73e5-e32e-4baf-8f4e-8ea21b7bd782 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:40.332262 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-824d73e5-e32e-4baf-8f4e-8ea21b7bd782 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:40.332776 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-824d73e5-e32e-4baf-8f4e-8ea21b7bd782 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:40.333561 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-824d73e5-e32e-4baf-8f4e-8ea21b7bd782 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:40.334030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-824d73e5-e32e-4baf-8f4e-8ea21b7bd782 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:40.334488 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-824d73e5-e32e-4baf-8f4e-8ea21b7bd782 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:40.337293 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:40.381186 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-824d73e5-e32e-4baf-8f4e-8ea21b7bd782 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063823 Sep 08 14:04:40.381782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1026/1938] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:40 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:40.414269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-48bd4652-8c21-454b-87e5-ee57c8a3f7f1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:40.425805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-48bd4652-8c21-454b-87e5-ee57c8a3f7f1 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.207063 Sep 08 14:04:40.426464 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 913/1939] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:40 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:40.470405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5121e01c-7ed7-40cf-bb05-b2cefec971e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:40.470997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5121e01c-7ed7-40cf-bb05-b2cefec971e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:40.478119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5121e01c-7ed7-40cf-bb05-b2cefec971e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:40.478991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5121e01c-7ed7-40cf-bb05-b2cefec971e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:40.479733 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5121e01c-7ed7-40cf-bb05-b2cefec971e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:40.480933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5121e01c-7ed7-40cf-bb05-b2cefec971e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:40.481688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5121e01c-7ed7-40cf-bb05-b2cefec971e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:40.482395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5121e01c-7ed7-40cf-bb05-b2cefec971e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:40.556555 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5121e01c-7ed7-40cf-bb05-b2cefec971e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:40.565302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5121e01c-7ed7-40cf-bb05-b2cefec971e5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.100713 Sep 08 14:04:40.565994 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1027/1940] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:40 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:40.674758 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-81cffb57-7b37-4f20-acf6-410c29f44385 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:40.676948 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-81cffb57-7b37-4f20-acf6-410c29f44385 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:40.684006 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-81cffb57-7b37-4f20-acf6-410c29f44385 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:40.684531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-81cffb57-7b37-4f20-acf6-410c29f44385 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:40.685030 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-81cffb57-7b37-4f20-acf6-410c29f44385 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:40.685765 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-81cffb57-7b37-4f20-acf6-410c29f44385 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:40.686264 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-81cffb57-7b37-4f20-acf6-410c29f44385 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:40.686694 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-81cffb57-7b37-4f20-acf6-410c29f44385 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:40.731873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-81cffb57-7b37-4f20-acf6-410c29f44385 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.063425 Sep 08 14:04:40.732375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 914/1941] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:40 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:41.268143 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a700a03c-3316-4265-9a89-0baa0b9613ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:41.270372 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a700a03c-3316-4265-9a89-0baa0b9613ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:41.278796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a700a03c-3316-4265-9a89-0baa0b9613ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:41.279515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a700a03c-3316-4265-9a89-0baa0b9613ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:41.280095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a700a03c-3316-4265-9a89-0baa0b9613ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:41.280832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a700a03c-3316-4265-9a89-0baa0b9613ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:41.282047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a700a03c-3316-4265-9a89-0baa0b9613ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:41.282047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a700a03c-3316-4265-9a89-0baa0b9613ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:41.316041 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a700a03c-3316-4265-9a89-0baa0b9613ce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051808 Sep 08 14:04:41.316626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1028/1942] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:41 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:41.396498 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7b8db060-1b5e-413f-b5eb-e766a575faad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:41.399278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7b8db060-1b5e-413f-b5eb-e766a575faad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:41.407045 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b8db060-1b5e-413f-b5eb-e766a575faad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:41.407528 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b8db060-1b5e-413f-b5eb-e766a575faad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:41.407957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b8db060-1b5e-413f-b5eb-e766a575faad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:41.408667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b8db060-1b5e-413f-b5eb-e766a575faad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:41.409114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b8db060-1b5e-413f-b5eb-e766a575faad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:41.409568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7b8db060-1b5e-413f-b5eb-e766a575faad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:41.454524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-73e5a72c-bf81-42e9-98b4-d83d864a6817 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:41.455411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-73e5a72c-bf81-42e9-98b4-d83d864a6817 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:41.458941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7b8db060-1b5e-413f-b5eb-e766a575faad tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066153 Sep 08 14:04:41.460611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 915/1943] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:41 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:41.474134 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73e5a72c-bf81-42e9-98b4-d83d864a6817 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:41.474745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73e5a72c-bf81-42e9-98b4-d83d864a6817 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:41.475206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73e5a72c-bf81-42e9-98b4-d83d864a6817 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:41.476091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73e5a72c-bf81-42e9-98b4-d83d864a6817 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:41.476587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73e5a72c-bf81-42e9-98b4-d83d864a6817 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:41.477074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73e5a72c-bf81-42e9-98b4-d83d864a6817 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:41.600205 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-795ae15d-c654-4821-84ca-00ad755e982a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:41.602746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-795ae15d-c654-4821-84ca-00ad755e982a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:41.613373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:41.615452 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-795ae15d-c654-4821-84ca-00ad755e982a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:41.616047 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-795ae15d-c654-4821-84ca-00ad755e982a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:41.616591 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-795ae15d-c654-4821-84ca-00ad755e982a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:41.618035 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-795ae15d-c654-4821-84ca-00ad755e982a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:41.619017 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-795ae15d-c654-4821-84ca-00ad755e982a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:41.620218 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-795ae15d-c654-4821-84ca-00ad755e982a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:41.693222 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-73e5a72c-bf81-42e9-98b4-d83d864a6817 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:41.709973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-73e5a72c-bf81-42e9-98b4-d83d864a6817 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.259440 Sep 08 14:04:41.710940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1029/1944] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:41 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:41.741519 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-795ae15d-c654-4821-84ca-00ad755e982a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:41.755303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d02534a8-5765-4c0c-9dbe-e6711147482d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:41.756103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d02534a8-5765-4c0c-9dbe-e6711147482d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:41.760392 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-795ae15d-c654-4821-84ca-00ad755e982a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.168581 Sep 08 14:04:41.765648 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 916/1945] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:41 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:41.770171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d02534a8-5765-4c0c-9dbe-e6711147482d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:41.771451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d02534a8-5765-4c0c-9dbe-e6711147482d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:41.772631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d02534a8-5765-4c0c-9dbe-e6711147482d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:41.774245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d02534a8-5765-4c0c-9dbe-e6711147482d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:41.774842 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d02534a8-5765-4c0c-9dbe-e6711147482d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:41.775469 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d02534a8-5765-4c0c-9dbe-e6711147482d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:41.825567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d02534a8-5765-4c0c-9dbe-e6711147482d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.083543 Sep 08 14:04:41.827028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1030/1946] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:41 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:04:42.336439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6a2e3aa5-e24e-434b-b744-0b297cb464a8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:42.340627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6a2e3aa5-e24e-434b-b744-0b297cb464a8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:42.351342 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a2e3aa5-e24e-434b-b744-0b297cb464a8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:42.352180 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a2e3aa5-e24e-434b-b744-0b297cb464a8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:42.352933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a2e3aa5-e24e-434b-b744-0b297cb464a8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:42.354770 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a2e3aa5-e24e-434b-b744-0b297cb464a8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:42.355608 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a2e3aa5-e24e-434b-b744-0b297cb464a8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:42.356368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a2e3aa5-e24e-434b-b744-0b297cb464a8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:42.396202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6a2e3aa5-e24e-434b-b744-0b297cb464a8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067257 Sep 08 14:04:42.397065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 917/1947] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:42 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:42.479291 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a9b8270d-e2b3-4aab-8f30-ec9a59ca04c8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:42.483118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a9b8270d-e2b3-4aab-8f30-ec9a59ca04c8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:42.494320 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a9b8270d-e2b3-4aab-8f30-ec9a59ca04c8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:42.494826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a9b8270d-e2b3-4aab-8f30-ec9a59ca04c8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:42.495268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a9b8270d-e2b3-4aab-8f30-ec9a59ca04c8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:42.495995 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a9b8270d-e2b3-4aab-8f30-ec9a59ca04c8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:42.496457 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a9b8270d-e2b3-4aab-8f30-ec9a59ca04c8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:42.496890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a9b8270d-e2b3-4aab-8f30-ec9a59ca04c8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:42.548289 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a9b8270d-e2b3-4aab-8f30-ec9a59ca04c8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.077074 Sep 08 14:04:42.549005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1031/1948] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:42 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:42.754176 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a5eb6649-35b9-45e8-b178-ae427c2d27c2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:42.755239 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a5eb6649-35b9-45e8-b178-ae427c2d27c2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:42.767597 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5eb6649-35b9-45e8-b178-ae427c2d27c2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:42.768425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5eb6649-35b9-45e8-b178-ae427c2d27c2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:42.769210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5eb6649-35b9-45e8-b178-ae427c2d27c2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:42.770458 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5eb6649-35b9-45e8-b178-ae427c2d27c2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:42.771524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5eb6649-35b9-45e8-b178-ae427c2d27c2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:42.772285 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5eb6649-35b9-45e8-b178-ae427c2d27c2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:42.797315 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5f30f0e6-f9cb-4d0b-b39a-392d2c1dfda0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:42.797963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5f30f0e6-f9cb-4d0b-b39a-392d2c1dfda0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:42.805804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f30f0e6-f9cb-4d0b-b39a-392d2c1dfda0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:42.806359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f30f0e6-f9cb-4d0b-b39a-392d2c1dfda0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:42.806359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f30f0e6-f9cb-4d0b-b39a-392d2c1dfda0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:42.807682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f30f0e6-f9cb-4d0b-b39a-392d2c1dfda0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:42.808216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f30f0e6-f9cb-4d0b-b39a-392d2c1dfda0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:42.808813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f30f0e6-f9cb-4d0b-b39a-392d2c1dfda0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:42.896214 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5f30f0e6-f9cb-4d0b-b39a-392d2c1dfda0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:42.913636 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5f30f0e6-f9cb-4d0b-b39a-392d2c1dfda0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.119900 Sep 08 14:04:42.915502 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1032/1949] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:42 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:42.924249 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-146c48b3-a68f-4e7d-bc89-abbdc03549b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:42.928044 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-146c48b3-a68f-4e7d-bc89-abbdc03549b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:42.938999 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-146c48b3-a68f-4e7d-bc89-abbdc03549b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:42.940354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-146c48b3-a68f-4e7d-bc89-abbdc03549b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:42.941518 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-146c48b3-a68f-4e7d-bc89-abbdc03549b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:42.945467 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-146c48b3-a68f-4e7d-bc89-abbdc03549b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:42.945467 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-146c48b3-a68f-4e7d-bc89-abbdc03549b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:42.945467 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-146c48b3-a68f-4e7d-bc89-abbdc03549b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:43.004789 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-146c48b3-a68f-4e7d-bc89-abbdc03549b3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.087454 Sep 08 14:04:43.005886 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1033/1950] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:42 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:43.019413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a5eb6649-35b9-45e8-b178-ae427c2d27c2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:43.042516 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a5eb6649-35b9-45e8-b178-ae427c2d27c2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.293493 Sep 08 14:04:43.042516 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 918/1951] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:42 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:43.421554 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f8c0b420-22a0-41c9-853b-fa733dacd0c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:43.425409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f8c0b420-22a0-41c9-853b-fa733dacd0c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:43.437624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8c0b420-22a0-41c9-853b-fa733dacd0c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:43.438881 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8c0b420-22a0-41c9-853b-fa733dacd0c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:43.439465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8c0b420-22a0-41c9-853b-fa733dacd0c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:43.440766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8c0b420-22a0-41c9-853b-fa733dacd0c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:43.441607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8c0b420-22a0-41c9-853b-fa733dacd0c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:43.442460 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8c0b420-22a0-41c9-853b-fa733dacd0c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:43.504448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f8c0b420-22a0-41c9-853b-fa733dacd0c0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.090433 Sep 08 14:04:43.505543 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1034/1952] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:43 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:43.565900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c207c4e2-feca-4148-8cde-a0553aa79478 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:43.569705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c207c4e2-feca-4148-8cde-a0553aa79478 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:43.581499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c207c4e2-feca-4148-8cde-a0553aa79478 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:43.582753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c207c4e2-feca-4148-8cde-a0553aa79478 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:43.583560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c207c4e2-feca-4148-8cde-a0553aa79478 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:43.584875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c207c4e2-feca-4148-8cde-a0553aa79478 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:43.585651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c207c4e2-feca-4148-8cde-a0553aa79478 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:43.586348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c207c4e2-feca-4148-8cde-a0553aa79478 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:43.631717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c207c4e2-feca-4148-8cde-a0553aa79478 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.072403 Sep 08 14:04:43.632326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 919/1953] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:43 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:43.954627 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ceb9c60a-88f8-4e4e-95f6-bea4399b3cbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:43.955746 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ceb9c60a-88f8-4e4e-95f6-bea4399b3cbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:43.968442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ceb9c60a-88f8-4e4e-95f6-bea4399b3cbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:43.969341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ceb9c60a-88f8-4e4e-95f6-bea4399b3cbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:43.970268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ceb9c60a-88f8-4e4e-95f6-bea4399b3cbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:43.971605 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ceb9c60a-88f8-4e4e-95f6-bea4399b3cbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:43.972498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ceb9c60a-88f8-4e4e-95f6-bea4399b3cbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:43.973516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ceb9c60a-88f8-4e4e-95f6-bea4399b3cbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:44.021943 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a6fc5c23-ee5f-4f21-9b68-850870b2045f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:44.024358 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a6fc5c23-ee5f-4f21-9b68-850870b2045f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:44.031799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6fc5c23-ee5f-4f21-9b68-850870b2045f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:44.032305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6fc5c23-ee5f-4f21-9b68-850870b2045f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:44.032740 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6fc5c23-ee5f-4f21-9b68-850870b2045f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:44.033461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6fc5c23-ee5f-4f21-9b68-850870b2045f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:44.033932 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6fc5c23-ee5f-4f21-9b68-850870b2045f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:44.034359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6fc5c23-ee5f-4f21-9b68-850870b2045f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:44.082052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ceb9c60a-88f8-4e4e-95f6-bea4399b3cbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:44.092563 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a6fc5c23-ee5f-4f21-9b68-850870b2045f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.074672 Sep 08 14:04:44.093723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 920/1954] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:44 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:04:44.097190 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ceb9c60a-88f8-4e4e-95f6-bea4399b3cbb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.150025 Sep 08 14:04:44.098284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1035/1955] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:43 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:44.098729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a9e8f4c3-158e-4473-99d8-c3b97ecd3935 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:44.099447 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a9e8f4c3-158e-4473-99d8-c3b97ecd3935 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:44.107490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a9e8f4c3-158e-4473-99d8-c3b97ecd3935 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:44.107490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a9e8f4c3-158e-4473-99d8-c3b97ecd3935 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:44.107490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a9e8f4c3-158e-4473-99d8-c3b97ecd3935 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:44.107490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a9e8f4c3-158e-4473-99d8-c3b97ecd3935 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:44.109782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a9e8f4c3-158e-4473-99d8-c3b97ecd3935 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:44.109782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a9e8f4c3-158e-4473-99d8-c3b97ecd3935 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:44.348579 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a9e8f4c3-158e-4473-99d8-c3b97ecd3935 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:44.360307 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a9e8f4c3-158e-4473-99d8-c3b97ecd3935 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.265251 Sep 08 14:04:44.360760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 921/1956] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:44 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:44.520786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-53f6ad27-e6db-4270-91f1-5648d32c2aa8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:44.523110 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-53f6ad27-e6db-4270-91f1-5648d32c2aa8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:44.530986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-53f6ad27-e6db-4270-91f1-5648d32c2aa8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:44.531482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-53f6ad27-e6db-4270-91f1-5648d32c2aa8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:44.531922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-53f6ad27-e6db-4270-91f1-5648d32c2aa8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:44.532637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-53f6ad27-e6db-4270-91f1-5648d32c2aa8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:44.533106 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-53f6ad27-e6db-4270-91f1-5648d32c2aa8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:44.533544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-53f6ad27-e6db-4270-91f1-5648d32c2aa8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:44.582856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-53f6ad27-e6db-4270-91f1-5648d32c2aa8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066161 Sep 08 14:04:44.583634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1036/1957] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:44 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:44.651680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-26b03ac3-4c02-4f90-a18f-565a767af02a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:44.653918 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-26b03ac3-4c02-4f90-a18f-565a767af02a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:44.660786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26b03ac3-4c02-4f90-a18f-565a767af02a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:44.661267 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26b03ac3-4c02-4f90-a18f-565a767af02a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:44.661705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26b03ac3-4c02-4f90-a18f-565a767af02a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:44.662442 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26b03ac3-4c02-4f90-a18f-565a767af02a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:44.662893 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26b03ac3-4c02-4f90-a18f-565a767af02a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:44.663314 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26b03ac3-4c02-4f90-a18f-565a767af02a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:44.668205 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:44.698878 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-26b03ac3-4c02-4f90-a18f-565a767af02a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053648 Sep 08 14:04:44.699815 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 922/1958] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:44 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:45.120730 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-495cea27-5ffc-448c-809a-4ef98d10ba84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:45.124408 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-495cea27-5ffc-448c-809a-4ef98d10ba84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:45.135524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-495cea27-5ffc-448c-809a-4ef98d10ba84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:45.136380 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-495cea27-5ffc-448c-809a-4ef98d10ba84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:45.137125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-424b50e7-5740-4fe4-8196-8a8b53dbce5a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:45.137824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-495cea27-5ffc-448c-809a-4ef98d10ba84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:45.137951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-424b50e7-5740-4fe4-8196-8a8b53dbce5a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:45.138425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-495cea27-5ffc-448c-809a-4ef98d10ba84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:45.139241 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-495cea27-5ffc-448c-809a-4ef98d10ba84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:45.139992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-495cea27-5ffc-448c-809a-4ef98d10ba84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:45.144953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-424b50e7-5740-4fe4-8196-8a8b53dbce5a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:45.145528 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-424b50e7-5740-4fe4-8196-8a8b53dbce5a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:45.146084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-424b50e7-5740-4fe4-8196-8a8b53dbce5a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:45.146834 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-424b50e7-5740-4fe4-8196-8a8b53dbce5a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:45.147294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-424b50e7-5740-4fe4-8196-8a8b53dbce5a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:45.147724 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-424b50e7-5740-4fe4-8196-8a8b53dbce5a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:45.197737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-495cea27-5ffc-448c-809a-4ef98d10ba84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.086873 Sep 08 14:04:45.198789 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1037/1959] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:45 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:45.220242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-424b50e7-5740-4fe4-8196-8a8b53dbce5a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:45.232984 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-424b50e7-5740-4fe4-8196-8a8b53dbce5a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.101213 Sep 08 14:04:45.233895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 923/1960] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:45 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:45.388404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-79a0cac1-fe4f-44d3-b589-ba62b3165d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:45.389275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-79a0cac1-fe4f-44d3-b589-ba62b3165d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:45.399534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-79a0cac1-fe4f-44d3-b589-ba62b3165d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:45.400436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-79a0cac1-fe4f-44d3-b589-ba62b3165d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:45.401219 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-79a0cac1-fe4f-44d3-b589-ba62b3165d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:45.403289 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-79a0cac1-fe4f-44d3-b589-ba62b3165d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:45.403603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-79a0cac1-fe4f-44d3-b589-ba62b3165d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:45.404389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-79a0cac1-fe4f-44d3-b589-ba62b3165d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:45.597940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8715edaa-85ae-4d96-9487-5c6466bb6b37 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:45.601298 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8715edaa-85ae-4d96-9487-5c6466bb6b37 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:45.604267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-79a0cac1-fe4f-44d3-b589-ba62b3165d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:45.610657 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8715edaa-85ae-4d96-9487-5c6466bb6b37 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:45.611202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8715edaa-85ae-4d96-9487-5c6466bb6b37 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:45.611642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8715edaa-85ae-4d96-9487-5c6466bb6b37 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:45.612364 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8715edaa-85ae-4d96-9487-5c6466bb6b37 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:45.612820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8715edaa-85ae-4d96-9487-5c6466bb6b37 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:45.613245 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8715edaa-85ae-4d96-9487-5c6466bb6b37 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:45.614261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-79a0cac1-fe4f-44d3-b589-ba62b3165d56 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.232669 Sep 08 14:04:45.614938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1038/1961] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:45 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:45.661201 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8715edaa-85ae-4d96-9487-5c6466bb6b37 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.068704 Sep 08 14:04:45.662014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 924/1962] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:45 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:45.716062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2ed55503-4a28-401f-8e9d-06787c68623e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:45.718532 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2ed55503-4a28-401f-8e9d-06787c68623e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:45.724942 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ed55503-4a28-401f-8e9d-06787c68623e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:45.725429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ed55503-4a28-401f-8e9d-06787c68623e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:45.725866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ed55503-4a28-401f-8e9d-06787c68623e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:45.726681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ed55503-4a28-401f-8e9d-06787c68623e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:45.727140 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ed55503-4a28-401f-8e9d-06787c68623e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:45.727568 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ed55503-4a28-401f-8e9d-06787c68623e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:45.763668 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2ed55503-4a28-401f-8e9d-06787c68623e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051824 Sep 08 14:04:45.764668 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1039/1963] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:45 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:46.214718 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7228dffc-05ee-4077-a1dd-2dced6a3fe57 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:46.217183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7228dffc-05ee-4077-a1dd-2dced6a3fe57 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:46.225863 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7228dffc-05ee-4077-a1dd-2dced6a3fe57 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:46.226611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7228dffc-05ee-4077-a1dd-2dced6a3fe57 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:46.227056 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7228dffc-05ee-4077-a1dd-2dced6a3fe57 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:46.227778 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7228dffc-05ee-4077-a1dd-2dced6a3fe57 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:46.229069 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7228dffc-05ee-4077-a1dd-2dced6a3fe57 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:46.229409 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7228dffc-05ee-4077-a1dd-2dced6a3fe57 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:46.269527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7e4e3022-64c2-4e5e-80e6-edefdbcc2d93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:46.270582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7e4e3022-64c2-4e5e-80e6-edefdbcc2d93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:46.273811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:46.274907 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7228dffc-05ee-4077-a1dd-2dced6a3fe57 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.064795 Sep 08 14:04:46.276549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 925/1964] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:46 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:46.283321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7e4e3022-64c2-4e5e-80e6-edefdbcc2d93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:46.284127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7e4e3022-64c2-4e5e-80e6-edefdbcc2d93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:46.284657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7e4e3022-64c2-4e5e-80e6-edefdbcc2d93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:46.285989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7e4e3022-64c2-4e5e-80e6-edefdbcc2d93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:46.286215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7e4e3022-64c2-4e5e-80e6-edefdbcc2d93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:46.286652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7e4e3022-64c2-4e5e-80e6-edefdbcc2d93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:46.360434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7e4e3022-64c2-4e5e-80e6-edefdbcc2d93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:46.368990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7e4e3022-64c2-4e5e-80e6-edefdbcc2d93 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.106054 Sep 08 14:04:46.369721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1040/1965] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:46 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:46.643468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-eb900189-1227-469f-a439-599535ac031b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:46.644091 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-eb900189-1227-469f-a439-599535ac031b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:46.650886 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb900189-1227-469f-a439-599535ac031b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:46.651363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb900189-1227-469f-a439-599535ac031b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:46.651793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb900189-1227-469f-a439-599535ac031b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:46.652491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb900189-1227-469f-a439-599535ac031b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:46.653072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb900189-1227-469f-a439-599535ac031b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:46.653761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb900189-1227-469f-a439-599535ac031b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:46.680506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-44e1a7b5-66d4-48ee-9d80-09d75d191715 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:46.684269 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-44e1a7b5-66d4-48ee-9d80-09d75d191715 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:46.695557 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44e1a7b5-66d4-48ee-9d80-09d75d191715 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:46.696501 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44e1a7b5-66d4-48ee-9d80-09d75d191715 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:46.697875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44e1a7b5-66d4-48ee-9d80-09d75d191715 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:46.699368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44e1a7b5-66d4-48ee-9d80-09d75d191715 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:46.700182 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44e1a7b5-66d4-48ee-9d80-09d75d191715 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:46.700944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-44e1a7b5-66d4-48ee-9d80-09d75d191715 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:46.747942 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-44e1a7b5-66d4-48ee-9d80-09d75d191715 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.074517 Sep 08 14:04:46.748799 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1041/1966] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:46 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:46.784000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d8b4a56c-2055-4903-9647-c7406b5250e1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:46.787559 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d8b4a56c-2055-4903-9647-c7406b5250e1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:46.797947 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d8b4a56c-2055-4903-9647-c7406b5250e1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:46.798616 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d8b4a56c-2055-4903-9647-c7406b5250e1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:46.799196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d8b4a56c-2055-4903-9647-c7406b5250e1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:46.800013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d8b4a56c-2055-4903-9647-c7406b5250e1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:46.800592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d8b4a56c-2055-4903-9647-c7406b5250e1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:46.801102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d8b4a56c-2055-4903-9647-c7406b5250e1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:46.831857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:46.838048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d8b4a56c-2055-4903-9647-c7406b5250e1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062824 Sep 08 14:04:46.839736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1042/1967] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:46 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:47.079807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-eb900189-1227-469f-a439-599535ac031b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:47.088867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-eb900189-1227-469f-a439-599535ac031b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.451707 Sep 08 14:04:47.089578 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 926/1968] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:46 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:47.295618 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-752bce9a-b7be-4bc8-970f-5b624a4038cd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:47.297829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-752bce9a-b7be-4bc8-970f-5b624a4038cd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:47.304444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-752bce9a-b7be-4bc8-970f-5b624a4038cd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:47.305009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-752bce9a-b7be-4bc8-970f-5b624a4038cd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:47.305523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-752bce9a-b7be-4bc8-970f-5b624a4038cd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:47.306958 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-752bce9a-b7be-4bc8-970f-5b624a4038cd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:47.307662 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-752bce9a-b7be-4bc8-970f-5b624a4038cd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:47.308327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-752bce9a-b7be-4bc8-970f-5b624a4038cd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:47.342125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-752bce9a-b7be-4bc8-970f-5b624a4038cd tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053568 Sep 08 14:04:47.342830 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1043/1969] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:47 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:47.394831 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-dde222d6-5a40-4ea6-88e6-bb703ad6699d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:47.395434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-dde222d6-5a40-4ea6-88e6-bb703ad6699d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:47.402391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dde222d6-5a40-4ea6-88e6-bb703ad6699d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:47.402892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dde222d6-5a40-4ea6-88e6-bb703ad6699d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:47.403356 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dde222d6-5a40-4ea6-88e6-bb703ad6699d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:47.404061 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dde222d6-5a40-4ea6-88e6-bb703ad6699d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:47.404521 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dde222d6-5a40-4ea6-88e6-bb703ad6699d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:47.404942 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dde222d6-5a40-4ea6-88e6-bb703ad6699d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:47.504648 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-dde222d6-5a40-4ea6-88e6-bb703ad6699d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:47.513755 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-dde222d6-5a40-4ea6-88e6-bb703ad6699d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.122893 Sep 08 14:04:47.514733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 927/1970] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:47 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:47.763272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d531bcca-d5c5-4d0f-865c-2a0aad0e3e8b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:47.765773 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d531bcca-d5c5-4d0f-865c-2a0aad0e3e8b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:47.776164 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d531bcca-d5c5-4d0f-865c-2a0aad0e3e8b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:47.776986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d531bcca-d5c5-4d0f-865c-2a0aad0e3e8b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:47.777679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d531bcca-d5c5-4d0f-865c-2a0aad0e3e8b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:47.778919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d531bcca-d5c5-4d0f-865c-2a0aad0e3e8b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:47.779693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d531bcca-d5c5-4d0f-865c-2a0aad0e3e8b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:47.780395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d531bcca-d5c5-4d0f-865c-2a0aad0e3e8b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:47.822730 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d531bcca-d5c5-4d0f-865c-2a0aad0e3e8b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063723 Sep 08 14:04:47.823412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1044/1971] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:47 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:47.856310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6bc157f5-61ca-4e3a-b5da-053bd30cb80e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:47.858596 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6bc157f5-61ca-4e3a-b5da-053bd30cb80e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:47.864951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6bc157f5-61ca-4e3a-b5da-053bd30cb80e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:47.865443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6bc157f5-61ca-4e3a-b5da-053bd30cb80e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:47.865883 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6bc157f5-61ca-4e3a-b5da-053bd30cb80e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:47.866623 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6bc157f5-61ca-4e3a-b5da-053bd30cb80e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:47.867077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6bc157f5-61ca-4e3a-b5da-053bd30cb80e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:47.867871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6bc157f5-61ca-4e3a-b5da-053bd30cb80e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:47.914393 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6bc157f5-61ca-4e3a-b5da-053bd30cb80e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062451 Sep 08 14:04:47.915782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 928/1972] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:47 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:48.118504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-aa37f0dc-ff5b-41b0-b37d-d1de400554ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:48.119512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-aa37f0dc-ff5b-41b0-b37d-d1de400554ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:48.129605 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa37f0dc-ff5b-41b0-b37d-d1de400554ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:48.130493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa37f0dc-ff5b-41b0-b37d-d1de400554ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:48.131210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa37f0dc-ff5b-41b0-b37d-d1de400554ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:48.132424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa37f0dc-ff5b-41b0-b37d-d1de400554ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:48.133241 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa37f0dc-ff5b-41b0-b37d-d1de400554ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:48.133976 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aa37f0dc-ff5b-41b0-b37d-d1de400554ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:48.358092 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-aa37f0dc-ff5b-41b0-b37d-d1de400554ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:48.362675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4aa48cb5-710a-4c25-a107-1326715ebe7a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:48.366432 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4aa48cb5-710a-4c25-a107-1326715ebe7a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:48.368048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-aa37f0dc-ff5b-41b0-b37d-d1de400554ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.256216 Sep 08 14:04:48.369434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1045/1973] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:48 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:48.378305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4aa48cb5-710a-4c25-a107-1326715ebe7a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:48.379224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4aa48cb5-710a-4c25-a107-1326715ebe7a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:48.380003 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4aa48cb5-710a-4c25-a107-1326715ebe7a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:48.381247 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4aa48cb5-710a-4c25-a107-1326715ebe7a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:48.382072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4aa48cb5-710a-4c25-a107-1326715ebe7a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:48.382895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4aa48cb5-710a-4c25-a107-1326715ebe7a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:48.425811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4aa48cb5-710a-4c25-a107-1326715ebe7a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.070891 Sep 08 14:04:48.427029 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 929/1974] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:48 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:48.543704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-85c82a0a-6a88-4c1c-852a-d2697e19e4b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:48.544732 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-85c82a0a-6a88-4c1c-852a-d2697e19e4b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:48.555452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85c82a0a-6a88-4c1c-852a-d2697e19e4b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:48.557517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85c82a0a-6a88-4c1c-852a-d2697e19e4b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:48.557764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85c82a0a-6a88-4c1c-852a-d2697e19e4b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:48.559042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85c82a0a-6a88-4c1c-852a-d2697e19e4b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:48.559875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85c82a0a-6a88-4c1c-852a-d2697e19e4b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:48.560652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-85c82a0a-6a88-4c1c-852a-d2697e19e4b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:48.656672 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-85c82a0a-6a88-4c1c-852a-d2697e19e4b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:48.669850 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-85c82a0a-6a88-4c1c-852a-d2697e19e4b0 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.133557 Sep 08 14:04:48.670485 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1046/1975] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:48 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:48.842585 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-41872f74-9dd5-489f-8772-b00ae96dbada tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:48.851398 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-41872f74-9dd5-489f-8772-b00ae96dbada tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:48.860282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-41872f74-9dd5-489f-8772-b00ae96dbada tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:48.861069 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-41872f74-9dd5-489f-8772-b00ae96dbada tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:48.867415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-41872f74-9dd5-489f-8772-b00ae96dbada tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:48.867415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-41872f74-9dd5-489f-8772-b00ae96dbada tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:48.867415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-41872f74-9dd5-489f-8772-b00ae96dbada tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:48.867415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-41872f74-9dd5-489f-8772-b00ae96dbada tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:48.908653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-41872f74-9dd5-489f-8772-b00ae96dbada tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.072867 Sep 08 14:04:48.909226 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 930/1976] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:48 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:48.935183 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5875a807-ead4-43ab-8105-21d0b07f79e7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:48.937431 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5875a807-ead4-43ab-8105-21d0b07f79e7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:48.944131 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5875a807-ead4-43ab-8105-21d0b07f79e7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:48.944622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5875a807-ead4-43ab-8105-21d0b07f79e7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:48.945062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5875a807-ead4-43ab-8105-21d0b07f79e7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:48.945761 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5875a807-ead4-43ab-8105-21d0b07f79e7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:48.946251 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5875a807-ead4-43ab-8105-21d0b07f79e7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:48.946684 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5875a807-ead4-43ab-8105-21d0b07f79e7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:48.986332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5875a807-ead4-43ab-8105-21d0b07f79e7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055664 Sep 08 14:04:48.987205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1047/1977] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:48 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:49.411902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1ad5f51b-0509-494c-ab10-b5a94b4a37f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:49.413021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1ad5f51b-0509-494c-ab10-b5a94b4a37f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:49.427002 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1ad5f51b-0509-494c-ab10-b5a94b4a37f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:49.428060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1ad5f51b-0509-494c-ab10-b5a94b4a37f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:49.428609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1ad5f51b-0509-494c-ab10-b5a94b4a37f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:49.429846 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1ad5f51b-0509-494c-ab10-b5a94b4a37f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:49.430690 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1ad5f51b-0509-494c-ab10-b5a94b4a37f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:49.432020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1ad5f51b-0509-494c-ab10-b5a94b4a37f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:49.445812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-66dc509e-fbaa-4203-a77b-245a486ca6ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:49.449029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-66dc509e-fbaa-4203-a77b-245a486ca6ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:49.457556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66dc509e-fbaa-4203-a77b-245a486ca6ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:49.458407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66dc509e-fbaa-4203-a77b-245a486ca6ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:49.459129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66dc509e-fbaa-4203-a77b-245a486ca6ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:49.460267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66dc509e-fbaa-4203-a77b-245a486ca6ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:49.461026 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66dc509e-fbaa-4203-a77b-245a486ca6ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:49.461707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66dc509e-fbaa-4203-a77b-245a486ca6ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:49.511031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-66dc509e-fbaa-4203-a77b-245a486ca6ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.072577 Sep 08 14:04:49.511716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1048/1978] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:49 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:04:49.628278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1ad5f51b-0509-494c-ab10-b5a94b4a37f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:49.642679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1ad5f51b-0509-494c-ab10-b5a94b4a37f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.237724 Sep 08 14:04:49.643740 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 931/1979] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:49 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:49.701462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7d0d73a2-98fb-47a8-8ee6-d676376a28bd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:49.702031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7d0d73a2-98fb-47a8-8ee6-d676376a28bd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:49.708796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7d0d73a2-98fb-47a8-8ee6-d676376a28bd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:49.709496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7d0d73a2-98fb-47a8-8ee6-d676376a28bd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:49.709990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7d0d73a2-98fb-47a8-8ee6-d676376a28bd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:49.710943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7d0d73a2-98fb-47a8-8ee6-d676376a28bd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:49.711485 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7d0d73a2-98fb-47a8-8ee6-d676376a28bd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:49.712020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7d0d73a2-98fb-47a8-8ee6-d676376a28bd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:49.816162 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7d0d73a2-98fb-47a8-8ee6-d676376a28bd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:49.830323 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7d0d73a2-98fb-47a8-8ee6-d676376a28bd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.132485 Sep 08 14:04:49.831345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1049/1980] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:49 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:49.925533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b5baee63-c870-4e45-ae1a-ae1033a5e646 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:49.927821 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b5baee63-c870-4e45-ae1a-ae1033a5e646 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:49.934308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b5baee63-c870-4e45-ae1a-ae1033a5e646 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:49.934966 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b5baee63-c870-4e45-ae1a-ae1033a5e646 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:49.935518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b5baee63-c870-4e45-ae1a-ae1033a5e646 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:49.936264 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b5baee63-c870-4e45-ae1a-ae1033a5e646 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:49.936761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b5baee63-c870-4e45-ae1a-ae1033a5e646 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:49.937226 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b5baee63-c870-4e45-ae1a-ae1033a5e646 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:49.970479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b5baee63-c870-4e45-ae1a-ae1033a5e646 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052138 Sep 08 14:04:49.971136 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 932/1981] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:49 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:50.007586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-328c9694-5f4e-4153-9323-85f978838583 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:50.011208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-328c9694-5f4e-4153-9323-85f978838583 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:50.022079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-328c9694-5f4e-4153-9323-85f978838583 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:50.022741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-328c9694-5f4e-4153-9323-85f978838583 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:50.023494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-328c9694-5f4e-4153-9323-85f978838583 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:50.025237 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-328c9694-5f4e-4153-9323-85f978838583 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:50.026065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-328c9694-5f4e-4153-9323-85f978838583 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:50.026886 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-328c9694-5f4e-4153-9323-85f978838583 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:50.083121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-328c9694-5f4e-4153-9323-85f978838583 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.082695 Sep 08 14:04:50.084217 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1050/1982] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:49 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:50.527611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cc6d5cd3-e3db-4f42-a4d2-525146c107f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:50.529774 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cc6d5cd3-e3db-4f42-a4d2-525146c107f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:50.536799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cc6d5cd3-e3db-4f42-a4d2-525146c107f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:50.537553 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cc6d5cd3-e3db-4f42-a4d2-525146c107f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:50.538000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cc6d5cd3-e3db-4f42-a4d2-525146c107f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:50.538829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cc6d5cd3-e3db-4f42-a4d2-525146c107f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:50.539385 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cc6d5cd3-e3db-4f42-a4d2-525146c107f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:50.539997 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cc6d5cd3-e3db-4f42-a4d2-525146c107f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:50.576341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cc6d5cd3-e3db-4f42-a4d2-525146c107f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053047 Sep 08 14:04:50.577182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 933/1983] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:50 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:04:50.679946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cf71ac6c-841c-4acd-8669-6782776bce18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:50.681548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cf71ac6c-841c-4acd-8669-6782776bce18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:50.692134 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cf71ac6c-841c-4acd-8669-6782776bce18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:50.692998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cf71ac6c-841c-4acd-8669-6782776bce18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:50.693776 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cf71ac6c-841c-4acd-8669-6782776bce18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:50.695035 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cf71ac6c-841c-4acd-8669-6782776bce18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:50.695837 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cf71ac6c-841c-4acd-8669-6782776bce18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:50.696663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cf71ac6c-841c-4acd-8669-6782776bce18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:50.863445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a76aa67f-f9e8-40d1-8586-c0545d8d1e28 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:50.864756 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a76aa67f-f9e8-40d1-8586-c0545d8d1e28 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:50.875826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cf71ac6c-841c-4acd-8669-6782776bce18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:50.878617 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a76aa67f-f9e8-40d1-8586-c0545d8d1e28 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:50.879277 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a76aa67f-f9e8-40d1-8586-c0545d8d1e28 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:50.879787 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a76aa67f-f9e8-40d1-8586-c0545d8d1e28 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:50.880688 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a76aa67f-f9e8-40d1-8586-c0545d8d1e28 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:50.881226 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a76aa67f-f9e8-40d1-8586-c0545d8d1e28 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:50.881971 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a76aa67f-f9e8-40d1-8586-c0545d8d1e28 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:50.890596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cf71ac6c-841c-4acd-8669-6782776bce18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.217552 Sep 08 14:04:50.891588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1051/1984] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:50 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:50.972202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a76aa67f-f9e8-40d1-8586-c0545d8d1e28 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:50.984859 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a76aa67f-f9e8-40d1-8586-c0545d8d1e28 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.128639 Sep 08 14:04:50.986188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 934/1985] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:50 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:50.988205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-84eb765a-7412-4e8b-9499-e3f093df0e39 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:50.990482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-84eb765a-7412-4e8b-9499-e3f093df0e39 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:50.997968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84eb765a-7412-4e8b-9499-e3f093df0e39 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:50.998497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84eb765a-7412-4e8b-9499-e3f093df0e39 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:50.998933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84eb765a-7412-4e8b-9499-e3f093df0e39 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:50.999631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84eb765a-7412-4e8b-9499-e3f093df0e39 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:51.000148 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84eb765a-7412-4e8b-9499-e3f093df0e39 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:51.000575 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84eb765a-7412-4e8b-9499-e3f093df0e39 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:51.053979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-84eb765a-7412-4e8b-9499-e3f093df0e39 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.072514 Sep 08 14:04:51.055019 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1052/1986] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:50 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:51.103676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bfa5b5dd-ad81-4deb-b01a-fcb4094ce11f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:51.106332 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bfa5b5dd-ad81-4deb-b01a-fcb4094ce11f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:51.114754 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bfa5b5dd-ad81-4deb-b01a-fcb4094ce11f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:51.115333 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bfa5b5dd-ad81-4deb-b01a-fcb4094ce11f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:51.115806 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bfa5b5dd-ad81-4deb-b01a-fcb4094ce11f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:51.116580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bfa5b5dd-ad81-4deb-b01a-fcb4094ce11f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:51.117229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bfa5b5dd-ad81-4deb-b01a-fcb4094ce11f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:51.117946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bfa5b5dd-ad81-4deb-b01a-fcb4094ce11f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:51.173602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bfa5b5dd-ad81-4deb-b01a-fcb4094ce11f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.075229 Sep 08 14:04:51.174641 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 935/1987] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:51 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:51.591656 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a4f80d8b-ebd2-4f86-836c-2a42996b294f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:51.594338 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a4f80d8b-ebd2-4f86-836c-2a42996b294f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:51.602876 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4f80d8b-ebd2-4f86-836c-2a42996b294f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:51.604010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4f80d8b-ebd2-4f86-836c-2a42996b294f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:51.604557 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4f80d8b-ebd2-4f86-836c-2a42996b294f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:51.605475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4f80d8b-ebd2-4f86-836c-2a42996b294f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:51.606185 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4f80d8b-ebd2-4f86-836c-2a42996b294f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:51.606817 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4f80d8b-ebd2-4f86-836c-2a42996b294f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:51.643251 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a4f80d8b-ebd2-4f86-836c-2a42996b294f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.056810 Sep 08 14:04:51.643936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1053/1988] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:51 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:04:51.932588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0783a574-17dc-481e-9515-516a1340c5c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:51.933676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0783a574-17dc-481e-9515-516a1340c5c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:51.944689 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0783a574-17dc-481e-9515-516a1340c5c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:51.945270 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0783a574-17dc-481e-9515-516a1340c5c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:51.946025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0783a574-17dc-481e-9515-516a1340c5c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:51.947806 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0783a574-17dc-481e-9515-516a1340c5c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:51.948603 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0783a574-17dc-481e-9515-516a1340c5c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:51.949349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0783a574-17dc-481e-9515-516a1340c5c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:52.027174 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-52a00cb9-a49a-4c27-8876-8aec5803e91a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:52.028436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-52a00cb9-a49a-4c27-8876-8aec5803e91a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:52.039398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-52a00cb9-a49a-4c27-8876-8aec5803e91a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:52.040444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-52a00cb9-a49a-4c27-8876-8aec5803e91a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:52.041000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-52a00cb9-a49a-4c27-8876-8aec5803e91a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:52.041804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-52a00cb9-a49a-4c27-8876-8aec5803e91a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:52.042302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-52a00cb9-a49a-4c27-8876-8aec5803e91a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:52.042758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-52a00cb9-a49a-4c27-8876-8aec5803e91a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:52.151409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-52a00cb9-a49a-4c27-8876-8aec5803e91a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:52.165053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-52a00cb9-a49a-4c27-8876-8aec5803e91a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.144682 Sep 08 14:04:52.166098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1054/1989] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:52 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 147 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:52.175030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bd2681f7-ce39-4cab-a618-ea51c242b3de tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:52.178517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bd2681f7-ce39-4cab-a618-ea51c242b3de tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:52.181299 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0783a574-17dc-481e-9515-516a1340c5c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:52.189024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd2681f7-ce39-4cab-a618-ea51c242b3de tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:52.189642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd2681f7-ce39-4cab-a618-ea51c242b3de tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:52.190348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd2681f7-ce39-4cab-a618-ea51c242b3de tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:52.191193 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd2681f7-ce39-4cab-a618-ea51c242b3de tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:52.191798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd2681f7-ce39-4cab-a618-ea51c242b3de tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:52.192513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd2681f7-ce39-4cab-a618-ea51c242b3de tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:52.199171 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0783a574-17dc-481e-9515-516a1340c5c9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.273317 Sep 08 14:04:52.200315 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 936/1990] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:51 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:52.210046 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-892f9c17-e48e-4e67-b9f0-1b7dc523196f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:52.214014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-892f9c17-e48e-4e67-b9f0-1b7dc523196f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:52.225221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-892f9c17-e48e-4e67-b9f0-1b7dc523196f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:52.226097 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-892f9c17-e48e-4e67-b9f0-1b7dc523196f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:52.226890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-892f9c17-e48e-4e67-b9f0-1b7dc523196f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:52.227877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bd2681f7-ce39-4cab-a618-ea51c242b3de tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059917 Sep 08 14:04:52.228366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-892f9c17-e48e-4e67-b9f0-1b7dc523196f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:52.228550 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1055/1991] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:52 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:52.229175 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-892f9c17-e48e-4e67-b9f0-1b7dc523196f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:52.229924 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-892f9c17-e48e-4e67-b9f0-1b7dc523196f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:52.301003 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-892f9c17-e48e-4e67-b9f0-1b7dc523196f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.098800 Sep 08 14:04:52.302015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 937/1992] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:52 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:52.657350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d13abd83-5341-4414-9353-5e688d696d84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:52.659573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d13abd83-5341-4414-9353-5e688d696d84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:52.666649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d13abd83-5341-4414-9353-5e688d696d84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:52.667184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d13abd83-5341-4414-9353-5e688d696d84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:52.667665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d13abd83-5341-4414-9353-5e688d696d84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:52.668460 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d13abd83-5341-4414-9353-5e688d696d84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:52.668968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d13abd83-5341-4414-9353-5e688d696d84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:52.669497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d13abd83-5341-4414-9353-5e688d696d84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:52.702511 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d13abd83-5341-4414-9353-5e688d696d84 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049266 Sep 08 14:04:52.703150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1056/1993] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:52 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:53.204893 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f9dd394e-7732-46a8-923d-58d653e89771 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:53.205688 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f9dd394e-7732-46a8-923d-58d653e89771 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:53.213431 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f9dd394e-7732-46a8-923d-58d653e89771 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:53.213923 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f9dd394e-7732-46a8-923d-58d653e89771 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:53.214401 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f9dd394e-7732-46a8-923d-58d653e89771 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:53.215107 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f9dd394e-7732-46a8-923d-58d653e89771 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:53.215561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f9dd394e-7732-46a8-923d-58d653e89771 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:53.215985 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f9dd394e-7732-46a8-923d-58d653e89771 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:53.237682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-722698b8-7305-4dd3-bcf0-0074cf91e8d7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:53.238768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-722698b8-7305-4dd3-bcf0-0074cf91e8d7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:53.250683 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-722698b8-7305-4dd3-bcf0-0074cf91e8d7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:53.251551 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-722698b8-7305-4dd3-bcf0-0074cf91e8d7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:53.252330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-722698b8-7305-4dd3-bcf0-0074cf91e8d7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:53.253569 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-722698b8-7305-4dd3-bcf0-0074cf91e8d7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:53.254436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-722698b8-7305-4dd3-bcf0-0074cf91e8d7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:53.255266 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-722698b8-7305-4dd3-bcf0-0074cf91e8d7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:53.292973 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f9dd394e-7732-46a8-923d-58d653e89771 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:53.296882 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:53.302482 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f9dd394e-7732-46a8-923d-58d653e89771 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.102283 Sep 08 14:04:53.303108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 938/1994] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:53 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:53.308338 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5d04290f-dbd8-4703-b423-c3148dac8526 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:53.310676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5d04290f-dbd8-4703-b423-c3148dac8526 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:53.317869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d04290f-dbd8-4703-b423-c3148dac8526 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:53.318306 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d04290f-dbd8-4703-b423-c3148dac8526 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:53.318821 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d04290f-dbd8-4703-b423-c3148dac8526 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:53.319460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d04290f-dbd8-4703-b423-c3148dac8526 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:53.319916 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d04290f-dbd8-4703-b423-c3148dac8526 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:53.320346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d04290f-dbd8-4703-b423-c3148dac8526 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:53.382227 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5d04290f-dbd8-4703-b423-c3148dac8526 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.077613 Sep 08 14:04:53.388805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 939/1995] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:53 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:53.395507 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1f5d47d6-ee4a-40f1-9f9b-8c86c5b2a0e0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:53.399202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1f5d47d6-ee4a-40f1-9f9b-8c86c5b2a0e0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:53.410368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f5d47d6-ee4a-40f1-9f9b-8c86c5b2a0e0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:53.411134 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f5d47d6-ee4a-40f1-9f9b-8c86c5b2a0e0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:53.411901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f5d47d6-ee4a-40f1-9f9b-8c86c5b2a0e0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:53.413147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f5d47d6-ee4a-40f1-9f9b-8c86c5b2a0e0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:53.413966 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f5d47d6-ee4a-40f1-9f9b-8c86c5b2a0e0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:53.414743 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f5d47d6-ee4a-40f1-9f9b-8c86c5b2a0e0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:53.444970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-722698b8-7305-4dd3-bcf0-0074cf91e8d7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:53.454558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-722698b8-7305-4dd3-bcf0-0074cf91e8d7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.224267 Sep 08 14:04:53.455357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1057/1996] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:53 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:53.471227 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1f5d47d6-ee4a-40f1-9f9b-8c86c5b2a0e0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.085584 Sep 08 14:04:53.472161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 940/1997] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:53 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:53.719504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4765d512-34a5-4a0e-b820-b64981102d1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:53.721882 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4765d512-34a5-4a0e-b820-b64981102d1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:53.728505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4765d512-34a5-4a0e-b820-b64981102d1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:53.729055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4765d512-34a5-4a0e-b820-b64981102d1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:53.729540 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4765d512-34a5-4a0e-b820-b64981102d1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:53.730432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4765d512-34a5-4a0e-b820-b64981102d1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:53.731021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4765d512-34a5-4a0e-b820-b64981102d1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:53.731507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4765d512-34a5-4a0e-b820-b64981102d1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:53.769747 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4765d512-34a5-4a0e-b820-b64981102d1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054768 Sep 08 14:04:53.770409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1058/1998] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:53 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:04:54.351879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-21e5f700-a90d-4a9a-8e36-296c6fd7b0ff tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:54.352772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-21e5f700-a90d-4a9a-8e36-296c6fd7b0ff tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:54.361852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21e5f700-a90d-4a9a-8e36-296c6fd7b0ff tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:54.362537 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21e5f700-a90d-4a9a-8e36-296c6fd7b0ff tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:54.363114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21e5f700-a90d-4a9a-8e36-296c6fd7b0ff tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:54.364163 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21e5f700-a90d-4a9a-8e36-296c6fd7b0ff tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:54.364759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21e5f700-a90d-4a9a-8e36-296c6fd7b0ff tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:54.365187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-21e5f700-a90d-4a9a-8e36-296c6fd7b0ff tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:54.398089 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d9939754-ef97-440c-863d-3da1afd052a5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:54.400454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d9939754-ef97-440c-863d-3da1afd052a5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:54.407239 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d9939754-ef97-440c-863d-3da1afd052a5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:54.407727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d9939754-ef97-440c-863d-3da1afd052a5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:54.408168 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d9939754-ef97-440c-863d-3da1afd052a5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:54.408864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d9939754-ef97-440c-863d-3da1afd052a5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:54.409317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d9939754-ef97-440c-863d-3da1afd052a5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:54.409741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d9939754-ef97-440c-863d-3da1afd052a5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:54.424825 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:54.443782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d9939754-ef97-440c-863d-3da1afd052a5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049800 Sep 08 14:04:54.444493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1059/1999] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:54 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:54.458414 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-21e5f700-a90d-4a9a-8e36-296c6fd7b0ff tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:54.467552 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-21e5f700-a90d-4a9a-8e36-296c6fd7b0ff tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.121434 Sep 08 14:04:54.468485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 941/2000] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:54 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:54.483010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c1da604a-639d-4394-b5cc-9ad021c74092 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:54.483750 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c1da604a-639d-4394-b5cc-9ad021c74092 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:54.490678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1da604a-639d-4394-b5cc-9ad021c74092 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:54.491262 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1da604a-639d-4394-b5cc-9ad021c74092 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:54.492605 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1da604a-639d-4394-b5cc-9ad021c74092 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:54.492802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f6a0b756-98e9-420c-ba6d-7aec9e24466f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:54.493924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1da604a-639d-4394-b5cc-9ad021c74092 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:54.494527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1da604a-639d-4394-b5cc-9ad021c74092 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:54.495035 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c1da604a-639d-4394-b5cc-9ad021c74092 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:54.497535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f6a0b756-98e9-420c-ba6d-7aec9e24466f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:54.508409 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f6a0b756-98e9-420c-ba6d-7aec9e24466f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:54.509836 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f6a0b756-98e9-420c-ba6d-7aec9e24466f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:54.510632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f6a0b756-98e9-420c-ba6d-7aec9e24466f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:54.511853 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f6a0b756-98e9-420c-ba6d-7aec9e24466f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:54.512559 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f6a0b756-98e9-420c-ba6d-7aec9e24466f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:54.513255 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f6a0b756-98e9-420c-ba6d-7aec9e24466f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:54.551880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f6a0b756-98e9-420c-ba6d-7aec9e24466f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067671 Sep 08 14:04:54.552539 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 942/2001] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:54 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:54.683976 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c1da604a-639d-4394-b5cc-9ad021c74092 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:54.692637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c1da604a-639d-4394-b5cc-9ad021c74092 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.215285 Sep 08 14:04:54.693273 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1060/2002] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:54 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:54.785551 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a377c95a-2671-4add-91cf-aa6da6b9d51c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:54.788165 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a377c95a-2671-4add-91cf-aa6da6b9d51c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:54.794746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a377c95a-2671-4add-91cf-aa6da6b9d51c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:54.795237 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a377c95a-2671-4add-91cf-aa6da6b9d51c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:54.795675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a377c95a-2671-4add-91cf-aa6da6b9d51c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:54.796381 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a377c95a-2671-4add-91cf-aa6da6b9d51c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:54.796835 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a377c95a-2671-4add-91cf-aa6da6b9d51c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:54.797258 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a377c95a-2671-4add-91cf-aa6da6b9d51c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:54.842719 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a377c95a-2671-4add-91cf-aa6da6b9d51c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061196 Sep 08 14:04:54.843701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 943/2003] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:54 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:55.462399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-012f7c8c-c070-44cd-8f27-764d0d88eabc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:55.466087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-012f7c8c-c070-44cd-8f27-764d0d88eabc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:55.478081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-012f7c8c-c070-44cd-8f27-764d0d88eabc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:55.479007 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-012f7c8c-c070-44cd-8f27-764d0d88eabc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:55.479819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-012f7c8c-c070-44cd-8f27-764d0d88eabc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:55.481107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-012f7c8c-c070-44cd-8f27-764d0d88eabc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:55.481930 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-012f7c8c-c070-44cd-8f27-764d0d88eabc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:55.482728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-012f7c8c-c070-44cd-8f27-764d0d88eabc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:55.524539 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-be88092b-b347-4775-a8f1-0e47470b88ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:55.525759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-be88092b-b347-4775-a8f1-0e47470b88ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:55.536519 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be88092b-b347-4775-a8f1-0e47470b88ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:55.537522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be88092b-b347-4775-a8f1-0e47470b88ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:55.538129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be88092b-b347-4775-a8f1-0e47470b88ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:55.538969 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be88092b-b347-4775-a8f1-0e47470b88ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:55.539739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be88092b-b347-4775-a8f1-0e47470b88ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:55.541028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-be88092b-b347-4775-a8f1-0e47470b88ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:55.542716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-012f7c8c-c070-44cd-8f27-764d0d88eabc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.087621 Sep 08 14:04:55.546473 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1061/2004] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:55 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:55.571895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-45a8dc74-7c82-4361-b274-6b0a20341254 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:55.575658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-45a8dc74-7c82-4361-b274-6b0a20341254 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:55.587130 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-45a8dc74-7c82-4361-b274-6b0a20341254 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:55.587454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-45a8dc74-7c82-4361-b274-6b0a20341254 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:55.588250 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-45a8dc74-7c82-4361-b274-6b0a20341254 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:55.589500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-45a8dc74-7c82-4361-b274-6b0a20341254 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:55.591815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-45a8dc74-7c82-4361-b274-6b0a20341254 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:55.592708 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-45a8dc74-7c82-4361-b274-6b0a20341254 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:55.636657 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-be88092b-b347-4775-a8f1-0e47470b88ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:55.644026 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-45a8dc74-7c82-4361-b274-6b0a20341254 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.079651 Sep 08 14:04:55.644607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1062/2005] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:55 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:55.648228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-be88092b-b347-4775-a8f1-0e47470b88ae tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.142304 Sep 08 14:04:55.648965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 944/2006] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:55 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:55.720830 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2b870c8e-fb6d-4b63-87c8-f8f7e85b0345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:55.721340 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2b870c8e-fb6d-4b63-87c8-f8f7e85b0345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:55.728365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b870c8e-fb6d-4b63-87c8-f8f7e85b0345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:55.728871 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b870c8e-fb6d-4b63-87c8-f8f7e85b0345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:55.729767 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b870c8e-fb6d-4b63-87c8-f8f7e85b0345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:55.730159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b870c8e-fb6d-4b63-87c8-f8f7e85b0345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:55.730625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b870c8e-fb6d-4b63-87c8-f8f7e85b0345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:55.731052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b870c8e-fb6d-4b63-87c8-f8f7e85b0345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:55.862823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b1961f89-6f22-42cb-a717-1896224f7943 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:55.866613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b1961f89-6f22-42cb-a717-1896224f7943 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:55.877666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1961f89-6f22-42cb-a717-1896224f7943 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:55.878449 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1961f89-6f22-42cb-a717-1896224f7943 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:55.879151 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1961f89-6f22-42cb-a717-1896224f7943 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:55.880221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1961f89-6f22-42cb-a717-1896224f7943 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:55.880924 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1961f89-6f22-42cb-a717-1896224f7943 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:55.881581 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1961f89-6f22-42cb-a717-1896224f7943 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:55.933991 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b1961f89-6f22-42cb-a717-1896224f7943 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.078304 Sep 08 14:04:55.934906 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 945/2007] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:55 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:04:55.954021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2b870c8e-fb6d-4b63-87c8-f8f7e85b0345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:55.967364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2b870c8e-fb6d-4b63-87c8-f8f7e85b0345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.251620 Sep 08 14:04:55.967786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1063/2008] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:55 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:56.566201 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9476f417-c7fc-43d3-a296-672ac003b882 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:56.569732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9476f417-c7fc-43d3-a296-672ac003b882 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:56.580968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9476f417-c7fc-43d3-a296-672ac003b882 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:56.581799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9476f417-c7fc-43d3-a296-672ac003b882 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:56.582581 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9476f417-c7fc-43d3-a296-672ac003b882 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:56.583673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9476f417-c7fc-43d3-a296-672ac003b882 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:56.584237 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9476f417-c7fc-43d3-a296-672ac003b882 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:56.584926 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9476f417-c7fc-43d3-a296-672ac003b882 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:56.635352 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9476f417-c7fc-43d3-a296-672ac003b882 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.075178 Sep 08 14:04:56.636340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 946/2009] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:56 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:56.658488 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5d182c7b-f65e-4040-9afe-847591c7ec46 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:56.660610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5d182c7b-f65e-4040-9afe-847591c7ec46 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:56.672348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5d182c7b-f65e-4040-9afe-847591c7ec46 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:56.673234 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5d182c7b-f65e-4040-9afe-847591c7ec46 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:56.674045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5d182c7b-f65e-4040-9afe-847591c7ec46 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:56.675346 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5d182c7b-f65e-4040-9afe-847591c7ec46 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:56.675730 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-beefd912-592c-46c8-9c2f-f43185599032 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:56.676178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5d182c7b-f65e-4040-9afe-847591c7ec46 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:56.676646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-beefd912-592c-46c8-9c2f-f43185599032 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:56.677139 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5d182c7b-f65e-4040-9afe-847591c7ec46 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:56.687411 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-beefd912-592c-46c8-9c2f-f43185599032 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:56.688223 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-beefd912-592c-46c8-9c2f-f43185599032 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:56.689015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-beefd912-592c-46c8-9c2f-f43185599032 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:56.690054 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-beefd912-592c-46c8-9c2f-f43185599032 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:56.690619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-beefd912-592c-46c8-9c2f-f43185599032 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:56.691329 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-beefd912-592c-46c8-9c2f-f43185599032 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:56.733619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5d182c7b-f65e-4040-9afe-847591c7ec46 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.079107 Sep 08 14:04:56.734969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1064/2010] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:56 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:56.780697 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-beefd912-592c-46c8-9c2f-f43185599032 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:56.789422 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-beefd912-592c-46c8-9c2f-f43185599032 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.120354 Sep 08 14:04:56.790757 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 947/2011] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:56 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:56.950979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-31ac4418-55e9-49df-abad-aaced4e787ee tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:56.954359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-31ac4418-55e9-49df-abad-aaced4e787ee tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:56.963941 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31ac4418-55e9-49df-abad-aaced4e787ee tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:56.964459 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31ac4418-55e9-49df-abad-aaced4e787ee tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:56.964909 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31ac4418-55e9-49df-abad-aaced4e787ee tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:56.965754 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31ac4418-55e9-49df-abad-aaced4e787ee tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:56.966439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31ac4418-55e9-49df-abad-aaced4e787ee tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:56.966905 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31ac4418-55e9-49df-abad-aaced4e787ee tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:56.998995 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-321cd611-b203-4aa9-83dc-0378e5e71be9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:56.999851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-321cd611-b203-4aa9-83dc-0378e5e71be9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:57.006676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-321cd611-b203-4aa9-83dc-0378e5e71be9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:57.007303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-321cd611-b203-4aa9-83dc-0378e5e71be9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:57.007871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-321cd611-b203-4aa9-83dc-0378e5e71be9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:57.009386 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-321cd611-b203-4aa9-83dc-0378e5e71be9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:57.009989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-321cd611-b203-4aa9-83dc-0378e5e71be9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:57.010589 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-321cd611-b203-4aa9-83dc-0378e5e71be9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:57.024999 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-31ac4418-55e9-49df-abad-aaced4e787ee tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.080252 Sep 08 14:04:57.025722 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1065/2012] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:56 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:57.188581 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-321cd611-b203-4aa9-83dc-0378e5e71be9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:57.198368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-321cd611-b203-4aa9-83dc-0378e5e71be9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.203444 Sep 08 14:04:57.199279 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 948/2013] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:56 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:57.654206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-02d2b058-71a8-4946-a3a1-62a762d9e269 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:57.656764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-02d2b058-71a8-4946-a3a1-62a762d9e269 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:57.663357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02d2b058-71a8-4946-a3a1-62a762d9e269 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:57.664507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02d2b058-71a8-4946-a3a1-62a762d9e269 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:57.664507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02d2b058-71a8-4946-a3a1-62a762d9e269 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:57.665664 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02d2b058-71a8-4946-a3a1-62a762d9e269 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:57.666254 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02d2b058-71a8-4946-a3a1-62a762d9e269 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:57.666607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-02d2b058-71a8-4946-a3a1-62a762d9e269 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:57.704617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-02d2b058-71a8-4946-a3a1-62a762d9e269 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057249 Sep 08 14:04:57.705302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1066/2014] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:57 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:57.750516 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0e8b0094-e9fa-417c-b4f4-67cb4e46ac44 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:57.752668 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0e8b0094-e9fa-417c-b4f4-67cb4e46ac44 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:57.759735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e8b0094-e9fa-417c-b4f4-67cb4e46ac44 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:57.760215 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e8b0094-e9fa-417c-b4f4-67cb4e46ac44 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:57.760644 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e8b0094-e9fa-417c-b4f4-67cb4e46ac44 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:57.761348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e8b0094-e9fa-417c-b4f4-67cb4e46ac44 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:57.761821 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e8b0094-e9fa-417c-b4f4-67cb4e46ac44 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:57.762219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0e8b0094-e9fa-417c-b4f4-67cb4e46ac44 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:57.794324 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0e8b0094-e9fa-417c-b4f4-67cb4e46ac44 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048010 Sep 08 14:04:57.795134 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 949/2015] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:57 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:57.815491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f22a9f58-b0b8-40ae-964f-9ed531fef0b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:57.816434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f22a9f58-b0b8-40ae-964f-9ed531fef0b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:57.825280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f22a9f58-b0b8-40ae-964f-9ed531fef0b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:57.825901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f22a9f58-b0b8-40ae-964f-9ed531fef0b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:57.826729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f22a9f58-b0b8-40ae-964f-9ed531fef0b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:57.827963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f22a9f58-b0b8-40ae-964f-9ed531fef0b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:57.828657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f22a9f58-b0b8-40ae-964f-9ed531fef0b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:57.829420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f22a9f58-b0b8-40ae-964f-9ed531fef0b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:57.917377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f22a9f58-b0b8-40ae-964f-9ed531fef0b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:57.932962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f22a9f58-b0b8-40ae-964f-9ed531fef0b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.122145 Sep 08 14:04:57.933939 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1067/2016] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:57 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:58.038751 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-58b86294-e223-4030-9431-58670cc19e10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:58.042109 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-58b86294-e223-4030-9431-58670cc19e10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:58.049297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-58b86294-e223-4030-9431-58670cc19e10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:58.049792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-58b86294-e223-4030-9431-58670cc19e10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:58.050271 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-58b86294-e223-4030-9431-58670cc19e10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:58.050989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-58b86294-e223-4030-9431-58670cc19e10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:58.051612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-58b86294-e223-4030-9431-58670cc19e10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:58.051959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-58b86294-e223-4030-9431-58670cc19e10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:58.088688 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-58b86294-e223-4030-9431-58670cc19e10 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054206 Sep 08 14:04:58.089292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 950/2017] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:58 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:58.223702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0d9429fb-7fe3-45a2-bd2b-992ea185eba2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:58.224213 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0d9429fb-7fe3-45a2-bd2b-992ea185eba2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:58.231752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d9429fb-7fe3-45a2-bd2b-992ea185eba2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:58.232504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d9429fb-7fe3-45a2-bd2b-992ea185eba2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:58.232970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d9429fb-7fe3-45a2-bd2b-992ea185eba2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:58.233695 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d9429fb-7fe3-45a2-bd2b-992ea185eba2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:58.234180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d9429fb-7fe3-45a2-bd2b-992ea185eba2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:58.234617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d9429fb-7fe3-45a2-bd2b-992ea185eba2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:58.408040 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0d9429fb-7fe3-45a2-bd2b-992ea185eba2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:58.422201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0d9429fb-7fe3-45a2-bd2b-992ea185eba2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.202352 Sep 08 14:04:58.423238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1068/2018] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:58 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:58.720746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c6eb86d4-1346-4f22-a977-db085e43e53d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:58.723749 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c6eb86d4-1346-4f22-a977-db085e43e53d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:58.731036 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6eb86d4-1346-4f22-a977-db085e43e53d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:58.731521 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6eb86d4-1346-4f22-a977-db085e43e53d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:58.731953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6eb86d4-1346-4f22-a977-db085e43e53d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:58.732663 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6eb86d4-1346-4f22-a977-db085e43e53d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:58.733110 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6eb86d4-1346-4f22-a977-db085e43e53d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:58.733616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6eb86d4-1346-4f22-a977-db085e43e53d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:58.770865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c6eb86d4-1346-4f22-a977-db085e43e53d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055901 Sep 08 14:04:58.771928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 951/2019] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:58 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:58.808232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0a2d0c88-8915-4a31-8c13-968f83fd5fe6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:58.810476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0a2d0c88-8915-4a31-8c13-968f83fd5fe6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:58.817408 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0a2d0c88-8915-4a31-8c13-968f83fd5fe6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:58.817909 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0a2d0c88-8915-4a31-8c13-968f83fd5fe6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:58.818392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0a2d0c88-8915-4a31-8c13-968f83fd5fe6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:58.819173 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0a2d0c88-8915-4a31-8c13-968f83fd5fe6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:58.819626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0a2d0c88-8915-4a31-8c13-968f83fd5fe6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:58.820065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0a2d0c88-8915-4a31-8c13-968f83fd5fe6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:58.857880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0a2d0c88-8915-4a31-8c13-968f83fd5fe6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054247 Sep 08 14:04:58.858543 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1069/2020] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:58 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:04:58.966213 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-349187c7-9aaf-496a-ac38-6b24da121ea1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:58.967213 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-349187c7-9aaf-496a-ac38-6b24da121ea1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:58.977872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-349187c7-9aaf-496a-ac38-6b24da121ea1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:58.978778 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-349187c7-9aaf-496a-ac38-6b24da121ea1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:58.979490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-349187c7-9aaf-496a-ac38-6b24da121ea1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:58.980680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-349187c7-9aaf-496a-ac38-6b24da121ea1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:58.981435 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-349187c7-9aaf-496a-ac38-6b24da121ea1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:58.982124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-349187c7-9aaf-496a-ac38-6b24da121ea1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:59.069366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-349187c7-9aaf-496a-ac38-6b24da121ea1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:59.079363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-349187c7-9aaf-496a-ac38-6b24da121ea1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.120840 Sep 08 14:04:59.079964 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 952/2021] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:58 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:04:59.102051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-360c3309-a91e-4c11-a18a-c5c7df2c2270 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:59.104675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-360c3309-a91e-4c11-a18a-c5c7df2c2270 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:59.111468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-360c3309-a91e-4c11-a18a-c5c7df2c2270 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:59.112010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-360c3309-a91e-4c11-a18a-c5c7df2c2270 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:59.112558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-360c3309-a91e-4c11-a18a-c5c7df2c2270 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:59.113232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-360c3309-a91e-4c11-a18a-c5c7df2c2270 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:59.113695 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-360c3309-a91e-4c11-a18a-c5c7df2c2270 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:59.114122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-360c3309-a91e-4c11-a18a-c5c7df2c2270 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:59.155153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-360c3309-a91e-4c11-a18a-c5c7df2c2270 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057661 Sep 08 14:04:59.155737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1070/2022] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:04:59 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:04:59.448646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-46c024d0-1cae-4ab6-97a0-de68e78d025f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:59.449272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-46c024d0-1cae-4ab6-97a0-de68e78d025f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:59.455728 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-46c024d0-1cae-4ab6-97a0-de68e78d025f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:59.456218 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-46c024d0-1cae-4ab6-97a0-de68e78d025f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:59.456659 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-46c024d0-1cae-4ab6-97a0-de68e78d025f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:59.457374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-46c024d0-1cae-4ab6-97a0-de68e78d025f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:59.457831 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-46c024d0-1cae-4ab6-97a0-de68e78d025f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:59.458304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-46c024d0-1cae-4ab6-97a0-de68e78d025f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:59.646829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-46c024d0-1cae-4ab6-97a0-de68e78d025f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:04:59.659124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-46c024d0-1cae-4ab6-97a0-de68e78d025f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.215342 Sep 08 14:04:59.659781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 953/2023] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:04:59 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:04:59.784646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-de6ff55f-6284-4855-9aea-66e50dd81b8c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:59.786964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-de6ff55f-6284-4855-9aea-66e50dd81b8c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:59.793857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de6ff55f-6284-4855-9aea-66e50dd81b8c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:59.794383 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de6ff55f-6284-4855-9aea-66e50dd81b8c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:59.794827 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de6ff55f-6284-4855-9aea-66e50dd81b8c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:59.795531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de6ff55f-6284-4855-9aea-66e50dd81b8c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:59.795988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de6ff55f-6284-4855-9aea-66e50dd81b8c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:59.796419 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de6ff55f-6284-4855-9aea-66e50dd81b8c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:59.831373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-de6ff55f-6284-4855-9aea-66e50dd81b8c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050834 Sep 08 14:04:59.832505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1071/2024] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:59 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:04:59.874854 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9860b16a-e971-4cbb-8ed5-26f7a8a46899 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:04:59.876989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9860b16a-e971-4cbb-8ed5-26f7a8a46899 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:04:59.884461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9860b16a-e971-4cbb-8ed5-26f7a8a46899 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:59.885078 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9860b16a-e971-4cbb-8ed5-26f7a8a46899 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:59.885612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9860b16a-e971-4cbb-8ed5-26f7a8a46899 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:59.886464 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9860b16a-e971-4cbb-8ed5-26f7a8a46899 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:04:59.887009 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9860b16a-e971-4cbb-8ed5-26f7a8a46899 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:04:59.887538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9860b16a-e971-4cbb-8ed5-26f7a8a46899 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:04:59.923155 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9860b16a-e971-4cbb-8ed5-26f7a8a46899 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053685 Sep 08 14:04:59.923894 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 954/2025] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:04:59 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:00.114592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c9284bfb-8e40-4629-a6a9-13c0dbe12c74 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:00.115571 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c9284bfb-8e40-4629-a6a9-13c0dbe12c74 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:00.126002 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c9284bfb-8e40-4629-a6a9-13c0dbe12c74 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:00.126599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c9284bfb-8e40-4629-a6a9-13c0dbe12c74 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:00.127113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c9284bfb-8e40-4629-a6a9-13c0dbe12c74 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:00.128270 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c9284bfb-8e40-4629-a6a9-13c0dbe12c74 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:00.129024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c9284bfb-8e40-4629-a6a9-13c0dbe12c74 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:00.129722 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c9284bfb-8e40-4629-a6a9-13c0dbe12c74 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:00.169473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7ffd3c51-eb85-4087-8730-209fb6bcc6b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:00.171800 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7ffd3c51-eb85-4087-8730-209fb6bcc6b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:00.178453 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7ffd3c51-eb85-4087-8730-209fb6bcc6b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:00.178939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7ffd3c51-eb85-4087-8730-209fb6bcc6b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:00.179372 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7ffd3c51-eb85-4087-8730-209fb6bcc6b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:00.180137 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7ffd3c51-eb85-4087-8730-209fb6bcc6b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:00.181335 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7ffd3c51-eb85-4087-8730-209fb6bcc6b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:00.182034 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7ffd3c51-eb85-4087-8730-209fb6bcc6b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:00.202117 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:00.216899 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7ffd3c51-eb85-4087-8730-209fb6bcc6b6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052431 Sep 08 14:05:00.217568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 955/2026] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:05:00 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:05:00.233484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c9284bfb-8e40-4629-a6a9-13c0dbe12c74 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:00.241844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c9284bfb-8e40-4629-a6a9-13c0dbe12c74 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.133375 Sep 08 14:05:00.243269 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1072/2027] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:00 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:00.687518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5985f9eb-36a6-45ab-a03c-f31a16d364f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:00.688631 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5985f9eb-36a6-45ab-a03c-f31a16d364f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:00.700766 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5985f9eb-36a6-45ab-a03c-f31a16d364f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:00.701511 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5985f9eb-36a6-45ab-a03c-f31a16d364f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:00.702254 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5985f9eb-36a6-45ab-a03c-f31a16d364f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:00.703464 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5985f9eb-36a6-45ab-a03c-f31a16d364f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:00.704593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5985f9eb-36a6-45ab-a03c-f31a16d364f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:00.705355 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5985f9eb-36a6-45ab-a03c-f31a16d364f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:00.847885 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6d3a490b-dc0b-498e-8638-46d61c1f70fc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:00.851325 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6d3a490b-dc0b-498e-8638-46d61c1f70fc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:00.861654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6d3a490b-dc0b-498e-8638-46d61c1f70fc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:00.862580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6d3a490b-dc0b-498e-8638-46d61c1f70fc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:00.863391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6d3a490b-dc0b-498e-8638-46d61c1f70fc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:00.864752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6d3a490b-dc0b-498e-8638-46d61c1f70fc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:00.865633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6d3a490b-dc0b-498e-8638-46d61c1f70fc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:00.866570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6d3a490b-dc0b-498e-8638-46d61c1f70fc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:00.903954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6d3a490b-dc0b-498e-8638-46d61c1f70fc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062333 Sep 08 14:05:00.904740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1073/2028] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:00 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:00.910658 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5985f9eb-36a6-45ab-a03c-f31a16d364f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:00.924132 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5985f9eb-36a6-45ab-a03c-f31a16d364f0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.243923 Sep 08 14:05:00.924731 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 956/2029] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:00 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:00.942948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-407e777c-9f36-417c-943c-dd193b4bc58e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:00.946739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-407e777c-9f36-417c-943c-dd193b4bc58e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:00.955809 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-407e777c-9f36-417c-943c-dd193b4bc58e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:00.956672 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-407e777c-9f36-417c-943c-dd193b4bc58e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:00.957447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-407e777c-9f36-417c-943c-dd193b4bc58e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:00.958712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-407e777c-9f36-417c-943c-dd193b4bc58e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:00.959503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-407e777c-9f36-417c-943c-dd193b4bc58e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:00.960202 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-407e777c-9f36-417c-943c-dd193b4bc58e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:01.010107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-407e777c-9f36-417c-943c-dd193b4bc58e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.073319 Sep 08 14:05:01.010971 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1074/2030] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:00 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:01.233329 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b2b375fa-a268-4573-8d9e-8f6895681c28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:01.235835 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b2b375fa-a268-4573-8d9e-8f6895681c28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:01.243064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b2b375fa-a268-4573-8d9e-8f6895681c28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:01.243685 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b2b375fa-a268-4573-8d9e-8f6895681c28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:01.244240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b2b375fa-a268-4573-8d9e-8f6895681c28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:01.245055 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b2b375fa-a268-4573-8d9e-8f6895681c28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:01.245615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b2b375fa-a268-4573-8d9e-8f6895681c28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:01.246154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b2b375fa-a268-4573-8d9e-8f6895681c28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:01.270854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9ddf71e4-d25b-47db-8f53-ce1dbcf808ca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:01.271469 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9ddf71e4-d25b-47db-8f53-ce1dbcf808ca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:01.286008 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b2b375fa-a268-4573-8d9e-8f6895681c28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.059369 Sep 08 14:05:01.286771 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 957/2031] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:05:01 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:05:01.291017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9ddf71e4-d25b-47db-8f53-ce1dbcf808ca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:01.291521 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9ddf71e4-d25b-47db-8f53-ce1dbcf808ca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:01.291962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9ddf71e4-d25b-47db-8f53-ce1dbcf808ca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:01.292681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9ddf71e4-d25b-47db-8f53-ce1dbcf808ca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:01.293139 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9ddf71e4-d25b-47db-8f53-ce1dbcf808ca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:01.293575 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9ddf71e4-d25b-47db-8f53-ce1dbcf808ca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:01.373189 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9ddf71e4-d25b-47db-8f53-ce1dbcf808ca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:01.381963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9ddf71e4-d25b-47db-8f53-ce1dbcf808ca tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.118158 Sep 08 14:05:01.382756 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1075/2032] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:01 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:01.918020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f7d86605-a5bf-41e7-935c-cbf072b1f3a0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:01.920591 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f7d86605-a5bf-41e7-935c-cbf072b1f3a0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:01.927858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f7d86605-a5bf-41e7-935c-cbf072b1f3a0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:01.928349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f7d86605-a5bf-41e7-935c-cbf072b1f3a0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:01.928793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f7d86605-a5bf-41e7-935c-cbf072b1f3a0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:01.929571 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f7d86605-a5bf-41e7-935c-cbf072b1f3a0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:01.930684 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f7d86605-a5bf-41e7-935c-cbf072b1f3a0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:01.931188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f7d86605-a5bf-41e7-935c-cbf072b1f3a0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:01.951000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0303b912-c752-433d-a81d-b46b3125827e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:01.952078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0303b912-c752-433d-a81d-b46b3125827e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:01.962426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0303b912-c752-433d-a81d-b46b3125827e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:01.963334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0303b912-c752-433d-a81d-b46b3125827e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:01.964037 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0303b912-c752-433d-a81d-b46b3125827e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:01.965734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0303b912-c752-433d-a81d-b46b3125827e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:01.966523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0303b912-c752-433d-a81d-b46b3125827e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:01.974450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0303b912-c752-433d-a81d-b46b3125827e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:01.977339 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f7d86605-a5bf-41e7-935c-cbf072b1f3a0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063742 Sep 08 14:05:01.978323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 958/2033] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:01 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:01.987965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:02.029780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d81e512f-c126-497f-a696-0669a091bb23 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:02.033714 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d81e512f-c126-497f-a696-0669a091bb23 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:02.044575 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d81e512f-c126-497f-a696-0669a091bb23 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:02.045454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d81e512f-c126-497f-a696-0669a091bb23 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:02.046270 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d81e512f-c126-497f-a696-0669a091bb23 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:02.047527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d81e512f-c126-497f-a696-0669a091bb23 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:02.048386 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d81e512f-c126-497f-a696-0669a091bb23 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:02.049162 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d81e512f-c126-497f-a696-0669a091bb23 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:02.102990 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d81e512f-c126-497f-a696-0669a091bb23 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.079113 Sep 08 14:05:02.104021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 959/2034] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:02 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:02.155880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0303b912-c752-433d-a81d-b46b3125827e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:02.167334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0303b912-c752-433d-a81d-b46b3125827e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.220994 Sep 08 14:05:02.168387 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1076/2035] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:01 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:02.303093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8fa44682-7e34-43ce-8951-96e2b552be5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:02.305901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8fa44682-7e34-43ce-8951-96e2b552be5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:02.312965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fa44682-7e34-43ce-8951-96e2b552be5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:02.313439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fa44682-7e34-43ce-8951-96e2b552be5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:02.313868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fa44682-7e34-43ce-8951-96e2b552be5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:02.314596 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fa44682-7e34-43ce-8951-96e2b552be5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:02.315041 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fa44682-7e34-43ce-8951-96e2b552be5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:02.315457 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fa44682-7e34-43ce-8951-96e2b552be5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:02.348513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8fa44682-7e34-43ce-8951-96e2b552be5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049677 Sep 08 14:05:02.350712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 960/2036] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:05:02 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:05:02.419686 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ac8f6665-e669-4752-a456-3676e8555636 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:02.420323 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ac8f6665-e669-4752-a456-3676e8555636 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:02.427196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ac8f6665-e669-4752-a456-3676e8555636 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:02.427746 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ac8f6665-e669-4752-a456-3676e8555636 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:02.428508 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ac8f6665-e669-4752-a456-3676e8555636 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:02.429320 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ac8f6665-e669-4752-a456-3676e8555636 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:02.430008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ac8f6665-e669-4752-a456-3676e8555636 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:02.431086 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ac8f6665-e669-4752-a456-3676e8555636 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:02.512792 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ac8f6665-e669-4752-a456-3676e8555636 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:02.521284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ac8f6665-e669-4752-a456-3676e8555636 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.105708 Sep 08 14:05:02.522018 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1077/2037] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:02 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:02.991139 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5e9472f6-6673-4982-8964-6c5c1dbbb81c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:02.993732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5e9472f6-6673-4982-8964-6c5c1dbbb81c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:03.002985 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e9472f6-6673-4982-8964-6c5c1dbbb81c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:03.003607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e9472f6-6673-4982-8964-6c5c1dbbb81c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:03.004136 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e9472f6-6673-4982-8964-6c5c1dbbb81c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:03.004935 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e9472f6-6673-4982-8964-6c5c1dbbb81c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:03.005593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e9472f6-6673-4982-8964-6c5c1dbbb81c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:03.006114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e9472f6-6673-4982-8964-6c5c1dbbb81c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:03.045957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5e9472f6-6673-4982-8964-6c5c1dbbb81c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058568 Sep 08 14:05:03.047021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 961/2038] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:02 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:03.128631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-75356978-f59b-4083-8a6b-0dcf08ebdf73 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:03.131627 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-75356978-f59b-4083-8a6b-0dcf08ebdf73 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:03.140316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75356978-f59b-4083-8a6b-0dcf08ebdf73 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:03.140931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75356978-f59b-4083-8a6b-0dcf08ebdf73 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:03.141528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75356978-f59b-4083-8a6b-0dcf08ebdf73 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:03.142403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75356978-f59b-4083-8a6b-0dcf08ebdf73 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:03.143055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75356978-f59b-4083-8a6b-0dcf08ebdf73 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:03.143566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75356978-f59b-4083-8a6b-0dcf08ebdf73 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:03.179729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-75356978-f59b-4083-8a6b-0dcf08ebdf73 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057840 Sep 08 14:05:03.180859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1078/2039] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:03 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:03.205375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fa944a24-130e-45cb-a3c8-63f9bd2fcf1f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:03.206016 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fa944a24-130e-45cb-a3c8-63f9bd2fcf1f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:03.213234 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa944a24-130e-45cb-a3c8-63f9bd2fcf1f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:03.213720 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa944a24-130e-45cb-a3c8-63f9bd2fcf1f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:03.214177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa944a24-130e-45cb-a3c8-63f9bd2fcf1f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:03.214888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa944a24-130e-45cb-a3c8-63f9bd2fcf1f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:03.215442 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa944a24-130e-45cb-a3c8-63f9bd2fcf1f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:03.215860 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa944a24-130e-45cb-a3c8-63f9bd2fcf1f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:03.367152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b10bfbd8-f8d0-4df2-afbd-808960e22bf5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:03.370193 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b10bfbd8-f8d0-4df2-afbd-808960e22bf5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:03.377322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b10bfbd8-f8d0-4df2-afbd-808960e22bf5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:03.383052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b10bfbd8-f8d0-4df2-afbd-808960e22bf5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:03.383052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b10bfbd8-f8d0-4df2-afbd-808960e22bf5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:03.383052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b10bfbd8-f8d0-4df2-afbd-808960e22bf5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:03.383052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b10bfbd8-f8d0-4df2-afbd-808960e22bf5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:03.384208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b10bfbd8-f8d0-4df2-afbd-808960e22bf5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:03.399097 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-fa944a24-130e-45cb-a3c8-63f9bd2fcf1f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:03.408243 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fa944a24-130e-45cb-a3c8-63f9bd2fcf1f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.207027 Sep 08 14:05:03.408843 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 962/2040] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:03 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:03.432712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b10bfbd8-f8d0-4df2-afbd-808960e22bf5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.072004 Sep 08 14:05:03.433760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1079/2041] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:05:03 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:05:03.556396 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d2af9a19-3c44-4d71-8179-09aec2c77547 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:03.557500 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d2af9a19-3c44-4d71-8179-09aec2c77547 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:03.565950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d2af9a19-3c44-4d71-8179-09aec2c77547 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:03.566472 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d2af9a19-3c44-4d71-8179-09aec2c77547 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:03.566909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d2af9a19-3c44-4d71-8179-09aec2c77547 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:03.567608 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d2af9a19-3c44-4d71-8179-09aec2c77547 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:03.568061 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d2af9a19-3c44-4d71-8179-09aec2c77547 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:03.568476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d2af9a19-3c44-4d71-8179-09aec2c77547 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:03.645048 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d2af9a19-3c44-4d71-8179-09aec2c77547 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:03.657491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d2af9a19-3c44-4d71-8179-09aec2c77547 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.107980 Sep 08 14:05:03.658469 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 963/2042] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:03 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:04.061693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4a9231fe-98f5-4247-9482-9784fb7b49a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:04.064026 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4a9231fe-98f5-4247-9482-9784fb7b49a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:04.070563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4a9231fe-98f5-4247-9482-9784fb7b49a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:04.071059 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4a9231fe-98f5-4247-9482-9784fb7b49a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:04.071496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4a9231fe-98f5-4247-9482-9784fb7b49a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:04.072227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4a9231fe-98f5-4247-9482-9784fb7b49a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:04.072704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4a9231fe-98f5-4247-9482-9784fb7b49a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:04.073215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4a9231fe-98f5-4247-9482-9784fb7b49a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:04.114603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4a9231fe-98f5-4247-9482-9784fb7b49a2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056969 Sep 08 14:05:04.115484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1080/2043] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:04 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:04.197459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ee5ce080-996e-4f0a-b628-4ddd6fe7438f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:04.200080 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ee5ce080-996e-4f0a-b628-4ddd6fe7438f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:04.206822 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ee5ce080-996e-4f0a-b628-4ddd6fe7438f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:04.207301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ee5ce080-996e-4f0a-b628-4ddd6fe7438f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:04.207727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ee5ce080-996e-4f0a-b628-4ddd6fe7438f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:04.208445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ee5ce080-996e-4f0a-b628-4ddd6fe7438f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:04.208872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ee5ce080-996e-4f0a-b628-4ddd6fe7438f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:04.209293 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ee5ce080-996e-4f0a-b628-4ddd6fe7438f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:04.253780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ee5ce080-996e-4f0a-b628-4ddd6fe7438f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061462 Sep 08 14:05:04.254298 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 964/2044] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:04 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:04.433418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d0ceadf5-2526-4558-8a92-bdfd60824c4b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:04.434269 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d0ceadf5-2526-4558-8a92-bdfd60824c4b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:04.440740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d0ceadf5-2526-4558-8a92-bdfd60824c4b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:04.441421 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d0ceadf5-2526-4558-8a92-bdfd60824c4b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:04.442254 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d0ceadf5-2526-4558-8a92-bdfd60824c4b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:04.443404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d0ceadf5-2526-4558-8a92-bdfd60824c4b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:04.444229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d0ceadf5-2526-4558-8a92-bdfd60824c4b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:04.445012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d0ceadf5-2526-4558-8a92-bdfd60824c4b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:04.450590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c3c9fac1-d006-448c-9fcd-8687f8517764 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:04.454246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c3c9fac1-d006-448c-9fcd-8687f8517764 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:04.464269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c3c9fac1-d006-448c-9fcd-8687f8517764 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:04.465510 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c3c9fac1-d006-448c-9fcd-8687f8517764 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:04.466031 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c3c9fac1-d006-448c-9fcd-8687f8517764 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:04.467144 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c3c9fac1-d006-448c-9fcd-8687f8517764 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:04.467955 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c3c9fac1-d006-448c-9fcd-8687f8517764 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:04.468611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c3c9fac1-d006-448c-9fcd-8687f8517764 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:04.525317 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c3c9fac1-d006-448c-9fcd-8687f8517764 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.080824 Sep 08 14:05:04.526116 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 965/2045] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:05:04 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:05:04.631283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d0ceadf5-2526-4558-8a92-bdfd60824c4b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:04.640918 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d0ceadf5-2526-4558-8a92-bdfd60824c4b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.211597 Sep 08 14:05:04.641506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1081/2046] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:04 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:04.694644 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-aaf6be78-9e72-4ec5-997c-40d141ee443b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:04.695848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-aaf6be78-9e72-4ec5-997c-40d141ee443b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:04.697792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:04.704565 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aaf6be78-9e72-4ec5-997c-40d141ee443b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:04.705415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aaf6be78-9e72-4ec5-997c-40d141ee443b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:04.706031 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aaf6be78-9e72-4ec5-997c-40d141ee443b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:04.706861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aaf6be78-9e72-4ec5-997c-40d141ee443b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:04.707416 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aaf6be78-9e72-4ec5-997c-40d141ee443b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:04.708039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aaf6be78-9e72-4ec5-997c-40d141ee443b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:04.778589 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-aaf6be78-9e72-4ec5-997c-40d141ee443b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:04.788018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-aaf6be78-9e72-4ec5-997c-40d141ee443b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.096938 Sep 08 14:05:04.788821 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 966/2047] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:04 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:05.128272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1160df1f-80a7-4c62-ae9b-7e22bae6a485 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:05.130778 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1160df1f-80a7-4c62-ae9b-7e22bae6a485 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:05.138489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1160df1f-80a7-4c62-ae9b-7e22bae6a485 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:05.140487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1160df1f-80a7-4c62-ae9b-7e22bae6a485 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:05.140487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1160df1f-80a7-4c62-ae9b-7e22bae6a485 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:05.140487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1160df1f-80a7-4c62-ae9b-7e22bae6a485 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:05.141137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1160df1f-80a7-4c62-ae9b-7e22bae6a485 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:05.141421 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1160df1f-80a7-4c62-ae9b-7e22bae6a485 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:05.175925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1160df1f-80a7-4c62-ae9b-7e22bae6a485 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051668 Sep 08 14:05:05.176532 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1082/2048] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:05 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:05.268225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3e0dfbf5-01b9-40dd-ba60-0c33d7f4833a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:05.270883 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3e0dfbf5-01b9-40dd-ba60-0c33d7f4833a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:05.285796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e0dfbf5-01b9-40dd-ba60-0c33d7f4833a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:05.286404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e0dfbf5-01b9-40dd-ba60-0c33d7f4833a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:05.286849 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e0dfbf5-01b9-40dd-ba60-0c33d7f4833a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:05.287576 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e0dfbf5-01b9-40dd-ba60-0c33d7f4833a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:05.288023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e0dfbf5-01b9-40dd-ba60-0c33d7f4833a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:05.288452 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3e0dfbf5-01b9-40dd-ba60-0c33d7f4833a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:05.324635 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3e0dfbf5-01b9-40dd-ba60-0c33d7f4833a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061525 Sep 08 14:05:05.325222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 967/2049] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:05 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:05.541717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b914bf2c-e5ab-4f70-b849-3d2a36028a59 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:05.543824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b914bf2c-e5ab-4f70-b849-3d2a36028a59 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:05.551048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b914bf2c-e5ab-4f70-b849-3d2a36028a59 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:05.551680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b914bf2c-e5ab-4f70-b849-3d2a36028a59 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:05.552230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b914bf2c-e5ab-4f70-b849-3d2a36028a59 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:05.553329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b914bf2c-e5ab-4f70-b849-3d2a36028a59 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:05.553835 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b914bf2c-e5ab-4f70-b849-3d2a36028a59 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:05.554357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b914bf2c-e5ab-4f70-b849-3d2a36028a59 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:05.590486 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b914bf2c-e5ab-4f70-b849-3d2a36028a59 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053663 Sep 08 14:05:05.591089 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1083/2050] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:05:05 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:05:05.667464 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9120f154-4dda-4082-982d-cf4b324b30a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:05.668220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9120f154-4dda-4082-982d-cf4b324b30a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:05.676199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9120f154-4dda-4082-982d-cf4b324b30a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:05.676847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9120f154-4dda-4082-982d-cf4b324b30a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:05.677378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9120f154-4dda-4082-982d-cf4b324b30a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:05.678830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9120f154-4dda-4082-982d-cf4b324b30a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:05.679709 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9120f154-4dda-4082-982d-cf4b324b30a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:05.680478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9120f154-4dda-4082-982d-cf4b324b30a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:05.820737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2e77a973-f235-4973-95ec-cdc44109abda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:05.821621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2e77a973-f235-4973-95ec-cdc44109abda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:05.830832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e77a973-f235-4973-95ec-cdc44109abda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:05.831729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e77a973-f235-4973-95ec-cdc44109abda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:05.832516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e77a973-f235-4973-95ec-cdc44109abda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:05.833690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e77a973-f235-4973-95ec-cdc44109abda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:05.834562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e77a973-f235-4973-95ec-cdc44109abda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:05.835107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e77a973-f235-4973-95ec-cdc44109abda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:05.851126 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9120f154-4dda-4082-982d-cf4b324b30a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:05.864644 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9120f154-4dda-4082-982d-cf4b324b30a3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.202214 Sep 08 14:05:05.865610 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 968/2051] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:05 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:05.931501 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2e77a973-f235-4973-95ec-cdc44109abda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:05.940512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2e77a973-f235-4973-95ec-cdc44109abda tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.126545 Sep 08 14:05:05.941119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1084/2052] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:05 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:06.194327 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-da7e0ce5-c6f4-455d-8064-ff9b5bd0df77 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:06.197863 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-da7e0ce5-c6f4-455d-8064-ff9b5bd0df77 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:06.207144 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da7e0ce5-c6f4-455d-8064-ff9b5bd0df77 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:06.207899 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da7e0ce5-c6f4-455d-8064-ff9b5bd0df77 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:06.208626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da7e0ce5-c6f4-455d-8064-ff9b5bd0df77 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:06.209863 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da7e0ce5-c6f4-455d-8064-ff9b5bd0df77 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:06.210713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da7e0ce5-c6f4-455d-8064-ff9b5bd0df77 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:06.212043 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-da7e0ce5-c6f4-455d-8064-ff9b5bd0df77 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:06.262164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-da7e0ce5-c6f4-455d-8064-ff9b5bd0df77 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.073607 Sep 08 14:05:06.262796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 969/2053] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:06 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:06.340306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4c75a1ca-5c93-4c38-b8a1-fc113e54e19f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:06.343671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4c75a1ca-5c93-4c38-b8a1-fc113e54e19f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:06.352170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4c75a1ca-5c93-4c38-b8a1-fc113e54e19f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:06.352922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4c75a1ca-5c93-4c38-b8a1-fc113e54e19f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:06.353689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4c75a1ca-5c93-4c38-b8a1-fc113e54e19f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:06.355003 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4c75a1ca-5c93-4c38-b8a1-fc113e54e19f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:06.355786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4c75a1ca-5c93-4c38-b8a1-fc113e54e19f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:06.356569 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4c75a1ca-5c93-4c38-b8a1-fc113e54e19f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:06.401126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4c75a1ca-5c93-4c38-b8a1-fc113e54e19f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067179 Sep 08 14:05:06.401858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1085/2054] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:06 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:06.613547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2eeb379c-e696-4a94-afe5-23be7bf93177 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:06.615970 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2eeb379c-e696-4a94-afe5-23be7bf93177 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:06.636282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2eeb379c-e696-4a94-afe5-23be7bf93177 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:06.637035 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2eeb379c-e696-4a94-afe5-23be7bf93177 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:06.637703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2eeb379c-e696-4a94-afe5-23be7bf93177 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:06.638983 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2eeb379c-e696-4a94-afe5-23be7bf93177 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:06.641745 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2eeb379c-e696-4a94-afe5-23be7bf93177 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:06.641745 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2eeb379c-e696-4a94-afe5-23be7bf93177 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:06.690838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2eeb379c-e696-4a94-afe5-23be7bf93177 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.089097 Sep 08 14:05:06.691433 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 970/2055] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:05:06 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:05:06.896834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fd06cb56-b42a-4e09-9606-d790e670247e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:06.897472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fd06cb56-b42a-4e09-9606-d790e670247e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:06.905135 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd06cb56-b42a-4e09-9606-d790e670247e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:06.905615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd06cb56-b42a-4e09-9606-d790e670247e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:06.906044 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd06cb56-b42a-4e09-9606-d790e670247e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:06.906783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd06cb56-b42a-4e09-9606-d790e670247e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:06.907230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd06cb56-b42a-4e09-9606-d790e670247e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:06.907650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd06cb56-b42a-4e09-9606-d790e670247e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:06.973308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-20c2f19b-a213-4c9e-a627-bfcb1065a8eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:06.974513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-20c2f19b-a213-4c9e-a627-bfcb1065a8eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:06.984594 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-20c2f19b-a213-4c9e-a627-bfcb1065a8eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:06.985130 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-20c2f19b-a213-4c9e-a627-bfcb1065a8eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:06.985713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-20c2f19b-a213-4c9e-a627-bfcb1065a8eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:06.986928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-20c2f19b-a213-4c9e-a627-bfcb1065a8eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:06.987755 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-20c2f19b-a213-4c9e-a627-bfcb1065a8eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:06.988483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-20c2f19b-a213-4c9e-a627-bfcb1065a8eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:07.081591 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-20c2f19b-a213-4c9e-a627-bfcb1065a8eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:07.092938 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-20c2f19b-a213-4c9e-a627-bfcb1065a8eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.129442 Sep 08 14:05:07.093542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 971/2056] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:06 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 131 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:07.095702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fd06cb56-b42a-4e09-9606-d790e670247e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:07.106745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fd06cb56-b42a-4e09-9606-d790e670247e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.214115 Sep 08 14:05:07.107448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1086/2057] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:06 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:07.281855 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-76dff650-b81e-43f0-9a07-84a123bd8eda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:07.284993 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-76dff650-b81e-43f0-9a07-84a123bd8eda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:07.294093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-76dff650-b81e-43f0-9a07-84a123bd8eda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:07.294899 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-76dff650-b81e-43f0-9a07-84a123bd8eda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:07.295344 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-76dff650-b81e-43f0-9a07-84a123bd8eda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:07.296272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-76dff650-b81e-43f0-9a07-84a123bd8eda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:07.296941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-76dff650-b81e-43f0-9a07-84a123bd8eda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:07.297625 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-76dff650-b81e-43f0-9a07-84a123bd8eda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:07.303281 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:07.339428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-76dff650-b81e-43f0-9a07-84a123bd8eda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064419 Sep 08 14:05:07.340012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 972/2058] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:07 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:07.416096 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-61d3bfd2-dd31-4bc5-91f4-079c6d2f1898 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:07.418908 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-61d3bfd2-dd31-4bc5-91f4-079c6d2f1898 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:07.427856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61d3bfd2-dd31-4bc5-91f4-079c6d2f1898 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:07.428362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61d3bfd2-dd31-4bc5-91f4-079c6d2f1898 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:07.428853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61d3bfd2-dd31-4bc5-91f4-079c6d2f1898 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:07.429689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61d3bfd2-dd31-4bc5-91f4-079c6d2f1898 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:07.430173 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61d3bfd2-dd31-4bc5-91f4-079c6d2f1898 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:07.430615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61d3bfd2-dd31-4bc5-91f4-079c6d2f1898 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:07.484074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-61d3bfd2-dd31-4bc5-91f4-079c6d2f1898 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.072636 Sep 08 14:05:07.485127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1087/2059] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:07 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:07.705645 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2eca7c67-5d1b-48c6-aab8-dba41182ef81 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:07.707938 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2eca7c67-5d1b-48c6-aab8-dba41182ef81 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:07.733545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2eca7c67-5d1b-48c6-aab8-dba41182ef81 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:07.733545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2eca7c67-5d1b-48c6-aab8-dba41182ef81 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:07.733545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2eca7c67-5d1b-48c6-aab8-dba41182ef81 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:07.733545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2eca7c67-5d1b-48c6-aab8-dba41182ef81 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:07.734414 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2eca7c67-5d1b-48c6-aab8-dba41182ef81 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:07.734414 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2eca7c67-5d1b-48c6-aab8-dba41182ef81 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:07.772009 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2eca7c67-5d1b-48c6-aab8-dba41182ef81 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.071877 Sep 08 14:05:07.772596 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 973/2060] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:05:07 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:05:08.128277 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-21c8973d-7633-4d6a-b6f2-9202770edca4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:08.129390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-21c8973d-7633-4d6a-b6f2-9202770edca4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:08.139972 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ce151785-a73a-4735-8495-f0b1710ffee6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:08.140888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-21c8973d-7633-4d6a-b6f2-9202770edca4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:08.141134 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ce151785-a73a-4735-8495-f0b1710ffee6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:08.141776 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-21c8973d-7633-4d6a-b6f2-9202770edca4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:08.142607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-21c8973d-7633-4d6a-b6f2-9202770edca4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:08.143844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-21c8973d-7633-4d6a-b6f2-9202770edca4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:08.144650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-21c8973d-7633-4d6a-b6f2-9202770edca4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:08.145679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-21c8973d-7633-4d6a-b6f2-9202770edca4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:08.152186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce151785-a73a-4735-8495-f0b1710ffee6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:08.152949 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce151785-a73a-4735-8495-f0b1710ffee6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:08.153637 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce151785-a73a-4735-8495-f0b1710ffee6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:08.154881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce151785-a73a-4735-8495-f0b1710ffee6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:08.155688 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce151785-a73a-4735-8495-f0b1710ffee6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:08.156443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ce151785-a73a-4735-8495-f0b1710ffee6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:08.250125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-21c8973d-7633-4d6a-b6f2-9202770edca4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:08.259148 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-21c8973d-7633-4d6a-b6f2-9202770edca4 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.137791 Sep 08 14:05:08.259977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1088/2061] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:08 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:08.349885 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ce151785-a73a-4735-8495-f0b1710ffee6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:08.353234 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-37f3d310-671b-420e-a3a9-9a42d18985ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:08.355470 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-37f3d310-671b-420e-a3a9-9a42d18985ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:08.362424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-37f3d310-671b-420e-a3a9-9a42d18985ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:08.362933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-37f3d310-671b-420e-a3a9-9a42d18985ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:08.363375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-37f3d310-671b-420e-a3a9-9a42d18985ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:08.364094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ce151785-a73a-4735-8495-f0b1710ffee6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.229559 Sep 08 14:05:08.364293 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-37f3d310-671b-420e-a3a9-9a42d18985ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:08.364678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-37f3d310-671b-420e-a3a9-9a42d18985ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:08.365613 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-37f3d310-671b-420e-a3a9-9a42d18985ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:08.365790 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 974/2062] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:08 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:08.408842 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-37f3d310-671b-420e-a3a9-9a42d18985ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059711 Sep 08 14:05:08.409878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1089/2063] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:08 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:08.499913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5bc85b6a-5a02-4253-b545-a52d0d58ee85 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:08.502091 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5bc85b6a-5a02-4253-b545-a52d0d58ee85 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:08.509975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5bc85b6a-5a02-4253-b545-a52d0d58ee85 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:08.510556 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5bc85b6a-5a02-4253-b545-a52d0d58ee85 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:08.511040 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5bc85b6a-5a02-4253-b545-a52d0d58ee85 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:08.511791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5bc85b6a-5a02-4253-b545-a52d0d58ee85 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:08.512286 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5bc85b6a-5a02-4253-b545-a52d0d58ee85 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:08.512753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5bc85b6a-5a02-4253-b545-a52d0d58ee85 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:08.545347 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5bc85b6a-5a02-4253-b545-a52d0d58ee85 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050159 Sep 08 14:05:08.545937 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 975/2064] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:08 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:08.786653 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a5e0263e-215e-4e03-b404-9dd7e0b56d5d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:08.790353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a5e0263e-215e-4e03-b404-9dd7e0b56d5d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:08.793535 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:08.801761 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a5e0263e-215e-4e03-b404-9dd7e0b56d5d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:08.810328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a5e0263e-215e-4e03-b404-9dd7e0b56d5d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:08.810328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a5e0263e-215e-4e03-b404-9dd7e0b56d5d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:08.810328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a5e0263e-215e-4e03-b404-9dd7e0b56d5d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:08.810328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a5e0263e-215e-4e03-b404-9dd7e0b56d5d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:08.811020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a5e0263e-215e-4e03-b404-9dd7e0b56d5d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:08.857521 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a5e0263e-215e-4e03-b404-9dd7e0b56d5d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.076261 Sep 08 14:05:08.858197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1090/2065] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:05:08 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:05:09.296478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d7d9a177-9158-46df-a8fb-e5a88744f931 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:09.297413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d7d9a177-9158-46df-a8fb-e5a88744f931 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:09.307567 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d7d9a177-9158-46df-a8fb-e5a88744f931 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:09.308413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d7d9a177-9158-46df-a8fb-e5a88744f931 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:09.309128 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d7d9a177-9158-46df-a8fb-e5a88744f931 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:09.310317 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d7d9a177-9158-46df-a8fb-e5a88744f931 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:09.311086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d7d9a177-9158-46df-a8fb-e5a88744f931 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:09.311801 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d7d9a177-9158-46df-a8fb-e5a88744f931 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:09.399403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d7d9a177-9158-46df-a8fb-e5a88744f931 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:09.402703 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bfb8702b-0b66-44c5-8d81-4b2a89a8e345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:09.403861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bfb8702b-0b66-44c5-8d81-4b2a89a8e345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:09.411428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d7d9a177-9158-46df-a8fb-e5a88744f931 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.118987 Sep 08 14:05:09.411725 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bfb8702b-0b66-44c5-8d81-4b2a89a8e345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:09.411915 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bfb8702b-0b66-44c5-8d81-4b2a89a8e345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:09.412226 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 976/2066] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:09 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:09.412402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bfb8702b-0b66-44c5-8d81-4b2a89a8e345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:09.413125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bfb8702b-0b66-44c5-8d81-4b2a89a8e345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:09.413651 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bfb8702b-0b66-44c5-8d81-4b2a89a8e345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:09.414682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bfb8702b-0b66-44c5-8d81-4b2a89a8e345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:09.424393 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3f26a4df-f944-41aa-8189-6d7c9c070f68 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:09.428002 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3f26a4df-f944-41aa-8189-6d7c9c070f68 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:09.438245 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3f26a4df-f944-41aa-8189-6d7c9c070f68 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:09.438578 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3f26a4df-f944-41aa-8189-6d7c9c070f68 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:09.439317 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3f26a4df-f944-41aa-8189-6d7c9c070f68 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:09.440492 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3f26a4df-f944-41aa-8189-6d7c9c070f68 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:09.441103 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3f26a4df-f944-41aa-8189-6d7c9c070f68 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:09.441800 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3f26a4df-f944-41aa-8189-6d7c9c070f68 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:09.481706 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3f26a4df-f944-41aa-8189-6d7c9c070f68 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063487 Sep 08 14:05:09.482361 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 977/2067] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:09 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:09.557879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-59b25156-c381-476f-9ef1-c2db16f16c5b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:09.560161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-59b25156-c381-476f-9ef1-c2db16f16c5b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:09.568023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59b25156-c381-476f-9ef1-c2db16f16c5b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:09.568520 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59b25156-c381-476f-9ef1-c2db16f16c5b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:09.568960 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59b25156-c381-476f-9ef1-c2db16f16c5b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:09.569735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59b25156-c381-476f-9ef1-c2db16f16c5b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:09.570300 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59b25156-c381-476f-9ef1-c2db16f16c5b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:09.570838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59b25156-c381-476f-9ef1-c2db16f16c5b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:09.600101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bfb8702b-0b66-44c5-8d81-4b2a89a8e345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:09.612440 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bfb8702b-0b66-44c5-8d81-4b2a89a8e345 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.216828 Sep 08 14:05:09.613031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1091/2068] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:09 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:09.624956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-59b25156-c381-476f-9ef1-c2db16f16c5b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.071264 Sep 08 14:05:09.626029 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 978/2069] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:09 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:09.871901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a1802c5b-ecd6-4460-9c01-ae302d636ad9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:09.874662 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a1802c5b-ecd6-4460-9c01-ae302d636ad9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:09.885159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1802c5b-ecd6-4460-9c01-ae302d636ad9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:09.886007 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1802c5b-ecd6-4460-9c01-ae302d636ad9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:09.886761 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1802c5b-ecd6-4460-9c01-ae302d636ad9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:09.888422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1802c5b-ecd6-4460-9c01-ae302d636ad9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:09.889023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1802c5b-ecd6-4460-9c01-ae302d636ad9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:09.889737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1802c5b-ecd6-4460-9c01-ae302d636ad9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:09.934191 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:09.943195 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a1802c5b-ecd6-4460-9c01-ae302d636ad9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.075079 Sep 08 14:05:09.950018 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1092/2070] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:05:09 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 08 14:05:10.444790 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b0ab3e96-d1e4-4734-8f99-14948b3aad04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:10.445914 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b0ab3e96-d1e4-4734-8f99-14948b3aad04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:10.454825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b0ab3e96-d1e4-4734-8f99-14948b3aad04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:10.455699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b0ab3e96-d1e4-4734-8f99-14948b3aad04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:10.456155 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b0ab3e96-d1e4-4734-8f99-14948b3aad04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:10.457283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b0ab3e96-d1e4-4734-8f99-14948b3aad04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:10.458062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b0ab3e96-d1e4-4734-8f99-14948b3aad04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:10.458794 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b0ab3e96-d1e4-4734-8f99-14948b3aad04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:10.502015 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-eec49a76-c87a-40b6-a824-b6435d2a5e57 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:10.505682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-eec49a76-c87a-40b6-a824-b6435d2a5e57 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:10.529123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eec49a76-c87a-40b6-a824-b6435d2a5e57 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:10.530040 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eec49a76-c87a-40b6-a824-b6435d2a5e57 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:10.530829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eec49a76-c87a-40b6-a824-b6435d2a5e57 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:10.532142 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eec49a76-c87a-40b6-a824-b6435d2a5e57 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:10.532973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eec49a76-c87a-40b6-a824-b6435d2a5e57 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:10.533712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eec49a76-c87a-40b6-a824-b6435d2a5e57 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:10.548493 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b0ab3e96-d1e4-4734-8f99-14948b3aad04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:10.558627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b0ab3e96-d1e4-4734-8f99-14948b3aad04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.121150 Sep 08 14:05:10.559225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 979/2071] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:10 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:10.583737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-eec49a76-c87a-40b6-a824-b6435d2a5e57 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.088746 Sep 08 14:05:10.584841 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1093/2072] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:10 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:10.638329 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0cc78d66-e2b7-4345-86ca-0e12453ec578 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:10.639558 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0cc78d66-e2b7-4345-86ca-0e12453ec578 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:10.642648 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fdc7527e-362d-4f84-be97-8edbc501128b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:10.645062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fdc7527e-362d-4f84-be97-8edbc501128b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:10.650273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0cc78d66-e2b7-4345-86ca-0e12453ec578 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:10.650959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0cc78d66-e2b7-4345-86ca-0e12453ec578 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:10.651510 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0cc78d66-e2b7-4345-86ca-0e12453ec578 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:10.651721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fdc7527e-362d-4f84-be97-8edbc501128b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:10.652221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fdc7527e-362d-4f84-be97-8edbc501128b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:10.652503 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0cc78d66-e2b7-4345-86ca-0e12453ec578 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:10.652741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fdc7527e-362d-4f84-be97-8edbc501128b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:10.653269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0cc78d66-e2b7-4345-86ca-0e12453ec578 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:10.653441 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fdc7527e-362d-4f84-be97-8edbc501128b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:10.653710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0cc78d66-e2b7-4345-86ca-0e12453ec578 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:10.653901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fdc7527e-362d-4f84-be97-8edbc501128b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:10.654373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fdc7527e-362d-4f84-be97-8edbc501128b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:10.691357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fdc7527e-362d-4f84-be97-8edbc501128b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053511 Sep 08 14:05:10.691944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1094/2073] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:10 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:10.829080 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0cc78d66-e2b7-4345-86ca-0e12453ec578 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:10.843682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0cc78d66-e2b7-4345-86ca-0e12453ec578 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.209908 Sep 08 14:05:10.844656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 980/2074] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:10 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:10.965046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ec5d03f1-a6bc-419c-b844-20cfe8247c15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:10.967240 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ec5d03f1-a6bc-419c-b844-20cfe8247c15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:10.978582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec5d03f1-a6bc-419c-b844-20cfe8247c15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:10.978582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec5d03f1-a6bc-419c-b844-20cfe8247c15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:10.978582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec5d03f1-a6bc-419c-b844-20cfe8247c15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:10.978582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec5d03f1-a6bc-419c-b844-20cfe8247c15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:10.979209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec5d03f1-a6bc-419c-b844-20cfe8247c15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:10.979209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ec5d03f1-a6bc-419c-b844-20cfe8247c15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:11.011354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ec5d03f1-a6bc-419c-b844-20cfe8247c15 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053809 Sep 08 14:05:11.013182 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1095/2075] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:05:10 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:05:11.599415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-04af71f8-2d94-4033-b8b5-ba60992eb12f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:11.600383 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-04af71f8-2d94-4033-b8b5-ba60992eb12f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:11.601764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-714c630b-3952-47a7-b4b3-983195b5ff5b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:11.605540 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-714c630b-3952-47a7-b4b3-983195b5ff5b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:11.610075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-04af71f8-2d94-4033-b8b5-ba60992eb12f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:11.610597 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-04af71f8-2d94-4033-b8b5-ba60992eb12f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:11.611041 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-04af71f8-2d94-4033-b8b5-ba60992eb12f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:11.611758 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-04af71f8-2d94-4033-b8b5-ba60992eb12f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:11.612214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-04af71f8-2d94-4033-b8b5-ba60992eb12f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:11.612637 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-04af71f8-2d94-4033-b8b5-ba60992eb12f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:11.617460 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-714c630b-3952-47a7-b4b3-983195b5ff5b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:11.618350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-714c630b-3952-47a7-b4b3-983195b5ff5b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:11.619091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-714c630b-3952-47a7-b4b3-983195b5ff5b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:11.620241 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-714c630b-3952-47a7-b4b3-983195b5ff5b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:11.621021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-714c630b-3952-47a7-b4b3-983195b5ff5b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:11.621684 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-714c630b-3952-47a7-b4b3-983195b5ff5b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:11.679354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-714c630b-3952-47a7-b4b3-983195b5ff5b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.085051 Sep 08 14:05:11.680661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1096/2076] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:11 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:11.698994 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-04af71f8-2d94-4033-b8b5-ba60992eb12f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:11.706964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7fcae627-7f25-4a38-8a5d-ef8688278adf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:11.709135 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7fcae627-7f25-4a38-8a5d-ef8688278adf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:11.709815 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-04af71f8-2d94-4033-b8b5-ba60992eb12f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.117049 Sep 08 14:05:11.711206 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 981/2077] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:11 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:11.716591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7fcae627-7f25-4a38-8a5d-ef8688278adf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:11.717136 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7fcae627-7f25-4a38-8a5d-ef8688278adf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:11.717590 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7fcae627-7f25-4a38-8a5d-ef8688278adf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:11.718417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7fcae627-7f25-4a38-8a5d-ef8688278adf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:11.718876 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7fcae627-7f25-4a38-8a5d-ef8688278adf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:11.719306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7fcae627-7f25-4a38-8a5d-ef8688278adf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:11.755644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7fcae627-7f25-4a38-8a5d-ef8688278adf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052029 Sep 08 14:05:11.755644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1097/2078] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:11 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:11.870044 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6115e8c9-48c0-499b-bd94-13dde8b046d2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:11.870573 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6115e8c9-48c0-499b-bd94-13dde8b046d2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:11.877094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6115e8c9-48c0-499b-bd94-13dde8b046d2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:11.877615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6115e8c9-48c0-499b-bd94-13dde8b046d2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:11.878061 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6115e8c9-48c0-499b-bd94-13dde8b046d2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:11.878857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6115e8c9-48c0-499b-bd94-13dde8b046d2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:11.879331 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6115e8c9-48c0-499b-bd94-13dde8b046d2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:11.879765 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6115e8c9-48c0-499b-bd94-13dde8b046d2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:12.026004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6ec92231-22cb-410a-b040-15990a0c9ffe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:05:12.029855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6ec92231-22cb-410a-b040-15990a0c9ffe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:12.037329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ec92231-22cb-410a-b040-15990a0c9ffe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:12.038089 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ec92231-22cb-410a-b040-15990a0c9ffe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:12.038771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ec92231-22cb-410a-b040-15990a0c9ffe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:12.039562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ec92231-22cb-410a-b040-15990a0c9ffe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:12.040136 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ec92231-22cb-410a-b040-15990a0c9ffe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:12.040805 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ec92231-22cb-410a-b040-15990a0c9ffe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:12.062963 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6115e8c9-48c0-499b-bd94-13dde8b046d2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:12.078758 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6115e8c9-48c0-499b-bd94-13dde8b046d2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.214162 Sep 08 14:05:12.079480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 982/2079] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:11 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:12.173962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6ec92231-22cb-410a-b040-15990a0c9ffe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "POST /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/action" status: 200 len: 37853 microversion: 2.20 time: 0.152922 Sep 08 14:05:12.175111 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1098/2080] 10.222.0.25 () {62 vars in 1319 bytes} [Thu Sep 8 14:05:12 2022] POST /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25/action => generated 37853 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 361 bytes (2 switches on core 0) Sep 08 14:05:12.696248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c6f0838c-2fdc-4399-a232-dccc2b907ec3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:12.698519 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c6f0838c-2fdc-4399-a232-dccc2b907ec3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:12.709705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6f0838c-2fdc-4399-a232-dccc2b907ec3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:12.710570 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6f0838c-2fdc-4399-a232-dccc2b907ec3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:12.711045 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6f0838c-2fdc-4399-a232-dccc2b907ec3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:12.712086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6f0838c-2fdc-4399-a232-dccc2b907ec3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:12.712660 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6f0838c-2fdc-4399-a232-dccc2b907ec3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:12.713194 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6f0838c-2fdc-4399-a232-dccc2b907ec3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:12.747943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b05334a7-b104-4e6f-a51d-12bdba61d13b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:12.748830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c6f0838c-2fdc-4399-a232-dccc2b907ec3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057016 Sep 08 14:05:12.749064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 983/2081] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:12 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:12.749238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b05334a7-b104-4e6f-a51d-12bdba61d13b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:12.758248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b05334a7-b104-4e6f-a51d-12bdba61d13b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:12.758755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b05334a7-b104-4e6f-a51d-12bdba61d13b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:12.759200 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b05334a7-b104-4e6f-a51d-12bdba61d13b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:12.759916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b05334a7-b104-4e6f-a51d-12bdba61d13b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:12.760383 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b05334a7-b104-4e6f-a51d-12bdba61d13b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:12.760818 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b05334a7-b104-4e6f-a51d-12bdba61d13b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:12.773145 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0a9fb688-fc39-4573-825e-01a1da2f42dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:12.775913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0a9fb688-fc39-4573-825e-01a1da2f42dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:12.782515 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0a9fb688-fc39-4573-825e-01a1da2f42dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:12.782990 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0a9fb688-fc39-4573-825e-01a1da2f42dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:12.783428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0a9fb688-fc39-4573-825e-01a1da2f42dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:12.784172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0a9fb688-fc39-4573-825e-01a1da2f42dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:12.784621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0a9fb688-fc39-4573-825e-01a1da2f42dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:12.785040 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0a9fb688-fc39-4573-825e-01a1da2f42dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:12.823042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0a9fb688-fc39-4573-825e-01a1da2f42dc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058146 Sep 08 14:05:12.824030 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 984/2082] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:12 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:12.849535 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b05334a7-b104-4e6f-a51d-12bdba61d13b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:12.861963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b05334a7-b104-4e6f-a51d-12bdba61d13b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.120899 Sep 08 14:05:12.862567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1099/2083] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:12 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:13.104882 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a7b3dc07-4417-466c-a5a3-8cbf21551898 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:13.105491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a7b3dc07-4417-466c-a5a3-8cbf21551898 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:13.112263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7b3dc07-4417-466c-a5a3-8cbf21551898 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:13.113014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7b3dc07-4417-466c-a5a3-8cbf21551898 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:13.113737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7b3dc07-4417-466c-a5a3-8cbf21551898 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:13.114465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7b3dc07-4417-466c-a5a3-8cbf21551898 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:13.115136 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7b3dc07-4417-466c-a5a3-8cbf21551898 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:13.115855 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7b3dc07-4417-466c-a5a3-8cbf21551898 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:13.301796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a7b3dc07-4417-466c-a5a3-8cbf21551898 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:13.312430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a7b3dc07-4417-466c-a5a3-8cbf21551898 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.212925 Sep 08 14:05:13.313456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 985/2084] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:13 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:13.768719 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6ac0b38c-efcc-494d-bd5c-9480ceace512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:13.772431 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6ac0b38c-efcc-494d-bd5c-9480ceace512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:13.784475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ac0b38c-efcc-494d-bd5c-9480ceace512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:13.785334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ac0b38c-efcc-494d-bd5c-9480ceace512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:13.786125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ac0b38c-efcc-494d-bd5c-9480ceace512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:13.787423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ac0b38c-efcc-494d-bd5c-9480ceace512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:13.788093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ac0b38c-efcc-494d-bd5c-9480ceace512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:13.788586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6ac0b38c-efcc-494d-bd5c-9480ceace512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:13.824835 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6ac0b38c-efcc-494d-bd5c-9480ceace512 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063448 Sep 08 14:05:13.825400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1100/2085] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:13 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:13.836752 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-88fcdde3-8f54-4142-be7c-4e1a7aa82e52 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:13.839002 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-88fcdde3-8f54-4142-be7c-4e1a7aa82e52 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:13.846299 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88fcdde3-8f54-4142-be7c-4e1a7aa82e52 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:13.846785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88fcdde3-8f54-4142-be7c-4e1a7aa82e52 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:13.847211 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88fcdde3-8f54-4142-be7c-4e1a7aa82e52 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:13.847904 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88fcdde3-8f54-4142-be7c-4e1a7aa82e52 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:13.848358 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88fcdde3-8f54-4142-be7c-4e1a7aa82e52 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:13.848777 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88fcdde3-8f54-4142-be7c-4e1a7aa82e52 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:13.883417 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-88fcdde3-8f54-4142-be7c-4e1a7aa82e52 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050403 Sep 08 14:05:13.883417 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 986/2086] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:13 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:13.903447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6328b0e9-ffba-40b7-8f4b-2ecbc97f9ab9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:13.903974 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6328b0e9-ffba-40b7-8f4b-2ecbc97f9ab9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:13.910770 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6328b0e9-ffba-40b7-8f4b-2ecbc97f9ab9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:13.911305 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6328b0e9-ffba-40b7-8f4b-2ecbc97f9ab9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:13.911794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6328b0e9-ffba-40b7-8f4b-2ecbc97f9ab9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:13.912447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6328b0e9-ffba-40b7-8f4b-2ecbc97f9ab9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:13.912897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6328b0e9-ffba-40b7-8f4b-2ecbc97f9ab9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:13.913357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6328b0e9-ffba-40b7-8f4b-2ecbc97f9ab9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:13.993539 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6328b0e9-ffba-40b7-8f4b-2ecbc97f9ab9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:14.004069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6328b0e9-ffba-40b7-8f4b-2ecbc97f9ab9 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.106276 Sep 08 14:05:14.004693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1101/2087] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:13 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:14.350473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d1d329e7-2f65-4550-baa2-e7699bc442b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:14.352052 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d1d329e7-2f65-4550-baa2-e7699bc442b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:14.363201 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d1d329e7-2f65-4550-baa2-e7699bc442b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:14.363986 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d1d329e7-2f65-4550-baa2-e7699bc442b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:14.365012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d1d329e7-2f65-4550-baa2-e7699bc442b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:14.365866 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d1d329e7-2f65-4550-baa2-e7699bc442b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:14.367481 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d1d329e7-2f65-4550-baa2-e7699bc442b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:14.368373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d1d329e7-2f65-4550-baa2-e7699bc442b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:14.545813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d1d329e7-2f65-4550-baa2-e7699bc442b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:14.560333 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d1d329e7-2f65-4550-baa2-e7699bc442b9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.216694 Sep 08 14:05:14.561302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 987/2088] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:14 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:14.838102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a6e395af-56d8-4bb9-8f67-50dc9e2482e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:14.840386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a6e395af-56d8-4bb9-8f67-50dc9e2482e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:14.847577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a6e395af-56d8-4bb9-8f67-50dc9e2482e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:14.848072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a6e395af-56d8-4bb9-8f67-50dc9e2482e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:14.848528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a6e395af-56d8-4bb9-8f67-50dc9e2482e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:14.849413 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a6e395af-56d8-4bb9-8f67-50dc9e2482e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:14.850385 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a6e395af-56d8-4bb9-8f67-50dc9e2482e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:14.851487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a6e395af-56d8-4bb9-8f67-50dc9e2482e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:14.886126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a6e395af-56d8-4bb9-8f67-50dc9e2482e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052960 Sep 08 14:05:14.886737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1102/2089] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:14 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:14.896665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-426bf8ea-87ad-46dd-bd79-b8613ae4c023 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:14.899763 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-426bf8ea-87ad-46dd-bd79-b8613ae4c023 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:14.907707 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-426bf8ea-87ad-46dd-bd79-b8613ae4c023 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:14.908247 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-426bf8ea-87ad-46dd-bd79-b8613ae4c023 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:14.908720 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-426bf8ea-87ad-46dd-bd79-b8613ae4c023 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:14.909469 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-426bf8ea-87ad-46dd-bd79-b8613ae4c023 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:14.909961 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-426bf8ea-87ad-46dd-bd79-b8613ae4c023 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:14.910554 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-426bf8ea-87ad-46dd-bd79-b8613ae4c023 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:14.939650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:14.945377 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-426bf8ea-87ad-46dd-bd79-b8613ae4c023 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052542 Sep 08 14:05:14.946045 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 988/2090] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:14 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:15.030969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-95939da3-d6fd-4f9b-bbad-501c345596b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:15.032031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-95939da3-d6fd-4f9b-bbad-501c345596b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:15.041582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-95939da3-d6fd-4f9b-bbad-501c345596b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:15.042503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-95939da3-d6fd-4f9b-bbad-501c345596b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:15.043055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-95939da3-d6fd-4f9b-bbad-501c345596b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:15.043874 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-95939da3-d6fd-4f9b-bbad-501c345596b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:15.044365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-95939da3-d6fd-4f9b-bbad-501c345596b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:15.044850 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-95939da3-d6fd-4f9b-bbad-501c345596b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:15.140865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-95939da3-d6fd-4f9b-bbad-501c345596b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:15.151481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-95939da3-d6fd-4f9b-bbad-501c345596b1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.126449 Sep 08 14:05:15.151703 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1103/2091] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:15 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:15.597334 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5d3645ec-aee2-4b07-8501-00347f310c0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:15.598480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5d3645ec-aee2-4b07-8501-00347f310c0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:15.607620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d3645ec-aee2-4b07-8501-00347f310c0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:15.608118 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d3645ec-aee2-4b07-8501-00347f310c0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:15.608560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d3645ec-aee2-4b07-8501-00347f310c0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:15.609304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d3645ec-aee2-4b07-8501-00347f310c0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:15.609767 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d3645ec-aee2-4b07-8501-00347f310c0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:15.610239 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d3645ec-aee2-4b07-8501-00347f310c0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:15.792799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5d3645ec-aee2-4b07-8501-00347f310c0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:15.806449 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5d3645ec-aee2-4b07-8501-00347f310c0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.215067 Sep 08 14:05:15.807303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 989/2092] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:15 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:15.902270 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ab3203c3-52f1-45c9-ae40-fcc97b944c85 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:15.905864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ab3203c3-52f1-45c9-ae40-fcc97b944c85 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:15.916718 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab3203c3-52f1-45c9-ae40-fcc97b944c85 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:15.917497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab3203c3-52f1-45c9-ae40-fcc97b944c85 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:15.918272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab3203c3-52f1-45c9-ae40-fcc97b944c85 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:15.919339 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab3203c3-52f1-45c9-ae40-fcc97b944c85 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:15.920018 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab3203c3-52f1-45c9-ae40-fcc97b944c85 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:15.920731 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab3203c3-52f1-45c9-ae40-fcc97b944c85 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:15.961489 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-19742966-8c39-404c-bbff-2afa23a2f4b0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:15.964459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-19742966-8c39-404c-bbff-2afa23a2f4b0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:15.969961 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ab3203c3-52f1-45c9-ae40-fcc97b944c85 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.074153 Sep 08 14:05:15.970699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1104/2093] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:15 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:15.980197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19742966-8c39-404c-bbff-2afa23a2f4b0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:15.980197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19742966-8c39-404c-bbff-2afa23a2f4b0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:15.980197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19742966-8c39-404c-bbff-2afa23a2f4b0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:15.980197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19742966-8c39-404c-bbff-2afa23a2f4b0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:15.982210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19742966-8c39-404c-bbff-2afa23a2f4b0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:15.982210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-19742966-8c39-404c-bbff-2afa23a2f4b0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:16.036497 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-19742966-8c39-404c-bbff-2afa23a2f4b0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.080702 Sep 08 14:05:16.037309 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 990/2094] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:15 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:16.189714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-01161b7f-c20b-4264-a0c4-a21cdbfac082 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:16.190905 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-01161b7f-c20b-4264-a0c4-a21cdbfac082 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:16.201146 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01161b7f-c20b-4264-a0c4-a21cdbfac082 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:16.201944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01161b7f-c20b-4264-a0c4-a21cdbfac082 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:16.202731 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01161b7f-c20b-4264-a0c4-a21cdbfac082 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:16.204253 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01161b7f-c20b-4264-a0c4-a21cdbfac082 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:16.205022 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01161b7f-c20b-4264-a0c4-a21cdbfac082 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:16.205808 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01161b7f-c20b-4264-a0c4-a21cdbfac082 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:16.294028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-01161b7f-c20b-4264-a0c4-a21cdbfac082 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:16.302675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-01161b7f-c20b-4264-a0c4-a21cdbfac082 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.118787 Sep 08 14:05:16.303277 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1105/2095] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:16 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:16.849486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e09a9567-591d-416f-bdb0-324630117c38 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:16.851413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e09a9567-591d-416f-bdb0-324630117c38 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:16.858692 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e09a9567-591d-416f-bdb0-324630117c38 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:16.859293 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e09a9567-591d-416f-bdb0-324630117c38 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:16.859776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e09a9567-591d-416f-bdb0-324630117c38 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:16.860538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e09a9567-591d-416f-bdb0-324630117c38 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:16.861061 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e09a9567-591d-416f-bdb0-324630117c38 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:16.861629 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e09a9567-591d-416f-bdb0-324630117c38 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:16.984988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-01642bbc-8c1c-46b5-b633-964444c64f81 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:16.987215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-01642bbc-8c1c-46b5-b633-964444c64f81 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:16.994262 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01642bbc-8c1c-46b5-b633-964444c64f81 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:16.994766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01642bbc-8c1c-46b5-b633-964444c64f81 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:16.995208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01642bbc-8c1c-46b5-b633-964444c64f81 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:16.995920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01642bbc-8c1c-46b5-b633-964444c64f81 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:16.996369 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01642bbc-8c1c-46b5-b633-964444c64f81 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:16.996794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01642bbc-8c1c-46b5-b633-964444c64f81 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:17.025141 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e09a9567-591d-416f-bdb0-324630117c38 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:17.036518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e09a9567-591d-416f-bdb0-324630117c38 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.193929 Sep 08 14:05:17.037438 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 991/2096] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:16 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:17.037667 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-01642bbc-8c1c-46b5-b633-964444c64f81 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056451 Sep 08 14:05:17.039059 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1106/2097] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:16 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:17.050706 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c0a5f581-8fdd-4501-9a32-fc65e6be819c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:17.053275 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c0a5f581-8fdd-4501-9a32-fc65e6be819c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:17.062916 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c0a5f581-8fdd-4501-9a32-fc65e6be819c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:17.063564 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c0a5f581-8fdd-4501-9a32-fc65e6be819c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:17.064137 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c0a5f581-8fdd-4501-9a32-fc65e6be819c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:17.064907 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c0a5f581-8fdd-4501-9a32-fc65e6be819c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:17.065479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c0a5f581-8fdd-4501-9a32-fc65e6be819c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:17.066011 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c0a5f581-8fdd-4501-9a32-fc65e6be819c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:17.100523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c0a5f581-8fdd-4501-9a32-fc65e6be819c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054427 Sep 08 14:05:17.101247 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 992/2098] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:17 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:17.327686 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4fa9e278-329b-4d68-9b94-c6b4625ceadb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:17.328202 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4fa9e278-329b-4d68-9b94-c6b4625ceadb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:17.334583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fa9e278-329b-4d68-9b94-c6b4625ceadb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:17.335707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fa9e278-329b-4d68-9b94-c6b4625ceadb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:17.336146 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fa9e278-329b-4d68-9b94-c6b4625ceadb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:17.336927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fa9e278-329b-4d68-9b94-c6b4625ceadb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:17.337384 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fa9e278-329b-4d68-9b94-c6b4625ceadb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:17.337839 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fa9e278-329b-4d68-9b94-c6b4625ceadb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:17.427405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4fa9e278-329b-4d68-9b94-c6b4625ceadb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:17.437899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4fa9e278-329b-4d68-9b94-c6b4625ceadb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.114322 Sep 08 14:05:17.438932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1107/2099] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:17 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:18.053675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d5d98e14-743b-4f0e-b613-8397c094a65e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:18.056961 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d5d98e14-743b-4f0e-b613-8397c094a65e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:18.067664 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d5d98e14-743b-4f0e-b613-8397c094a65e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:18.068780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d5d98e14-743b-4f0e-b613-8397c094a65e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:18.069312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d5d98e14-743b-4f0e-b613-8397c094a65e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:18.070070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d5d98e14-743b-4f0e-b613-8397c094a65e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:18.070560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d5d98e14-743b-4f0e-b613-8397c094a65e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:18.070982 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d5d98e14-743b-4f0e-b613-8397c094a65e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:18.073572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c2f9076a-30bf-47d3-b66c-ce98d05dd727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:18.074571 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c2f9076a-30bf-47d3-b66c-ce98d05dd727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:18.081549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2f9076a-30bf-47d3-b66c-ce98d05dd727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:18.082042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2f9076a-30bf-47d3-b66c-ce98d05dd727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:18.082552 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2f9076a-30bf-47d3-b66c-ce98d05dd727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:18.083286 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2f9076a-30bf-47d3-b66c-ce98d05dd727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:18.083741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2f9076a-30bf-47d3-b66c-ce98d05dd727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:18.084260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c2f9076a-30bf-47d3-b66c-ce98d05dd727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:18.108401 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d5d98e14-743b-4f0e-b613-8397c094a65e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058833 Sep 08 14:05:18.109753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 993/2100] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:18 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:18.115226 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-95859167-0130-4db8-9a15-add6e6c03d7f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:18.117680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-95859167-0130-4db8-9a15-add6e6c03d7f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:18.124560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-95859167-0130-4db8-9a15-add6e6c03d7f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:18.125173 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-95859167-0130-4db8-9a15-add6e6c03d7f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:18.125624 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-95859167-0130-4db8-9a15-add6e6c03d7f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:18.126357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-95859167-0130-4db8-9a15-add6e6c03d7f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:18.127070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-95859167-0130-4db8-9a15-add6e6c03d7f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:18.127261 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-95859167-0130-4db8-9a15-add6e6c03d7f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:18.167165 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-95859167-0130-4db8-9a15-add6e6c03d7f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056018 Sep 08 14:05:18.168162 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 994/2101] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:18 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:18.267700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c2f9076a-30bf-47d3-b66c-ce98d05dd727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:18.277237 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c2f9076a-30bf-47d3-b66c-ce98d05dd727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.208616 Sep 08 14:05:18.277982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1108/2102] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:18 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:18.463298 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fce7c29d-781c-49d0-8083-fa43e530f0cf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:18.464259 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fce7c29d-781c-49d0-8083-fa43e530f0cf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:18.470429 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fce7c29d-781c-49d0-8083-fa43e530f0cf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:18.470911 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fce7c29d-781c-49d0-8083-fa43e530f0cf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:18.471422 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fce7c29d-781c-49d0-8083-fa43e530f0cf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:18.472306 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fce7c29d-781c-49d0-8083-fa43e530f0cf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:18.472796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fce7c29d-781c-49d0-8083-fa43e530f0cf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:18.473187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fce7c29d-781c-49d0-8083-fa43e530f0cf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:18.559175 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-fce7c29d-781c-49d0-8083-fa43e530f0cf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:18.572419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fce7c29d-781c-49d0-8083-fa43e530f0cf tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.113505 Sep 08 14:05:18.573210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 995/2103] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:18 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:19.126726 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a1744742-9fb8-48f3-ad09-b5e98b59618c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:19.129585 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a1744742-9fb8-48f3-ad09-b5e98b59618c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:19.140207 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1744742-9fb8-48f3-ad09-b5e98b59618c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:19.141093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1744742-9fb8-48f3-ad09-b5e98b59618c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:19.141883 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1744742-9fb8-48f3-ad09-b5e98b59618c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:19.143087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1744742-9fb8-48f3-ad09-b5e98b59618c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:19.143900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1744742-9fb8-48f3-ad09-b5e98b59618c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:19.144635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1744742-9fb8-48f3-ad09-b5e98b59618c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:19.176668 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:19.184598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a1744742-9fb8-48f3-ad09-b5e98b59618c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064630 Sep 08 14:05:19.185184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1109/2104] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:19 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:19.188137 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-59f86296-54c0-4d34-aedf-4c477242c286 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:19.191669 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-59f86296-54c0-4d34-aedf-4c477242c286 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:19.203143 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59f86296-54c0-4d34-aedf-4c477242c286 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:19.203975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59f86296-54c0-4d34-aedf-4c477242c286 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:19.204725 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59f86296-54c0-4d34-aedf-4c477242c286 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:19.205925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59f86296-54c0-4d34-aedf-4c477242c286 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:19.206514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59f86296-54c0-4d34-aedf-4c477242c286 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:19.207047 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59f86296-54c0-4d34-aedf-4c477242c286 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:19.250673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-59f86296-54c0-4d34-aedf-4c477242c286 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069943 Sep 08 14:05:19.251149 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 996/2105] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:19 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:19.301855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3eb9a6ea-68b8-46e0-8806-a104c362aabe tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:19.302498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3eb9a6ea-68b8-46e0-8806-a104c362aabe tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:19.316160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3eb9a6ea-68b8-46e0-8806-a104c362aabe tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:19.316790 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3eb9a6ea-68b8-46e0-8806-a104c362aabe tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:19.317232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3eb9a6ea-68b8-46e0-8806-a104c362aabe tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:19.317935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3eb9a6ea-68b8-46e0-8806-a104c362aabe tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:19.318424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3eb9a6ea-68b8-46e0-8806-a104c362aabe tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:19.318838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3eb9a6ea-68b8-46e0-8806-a104c362aabe tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:19.499264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3eb9a6ea-68b8-46e0-8806-a104c362aabe tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:19.508173 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3eb9a6ea-68b8-46e0-8806-a104c362aabe tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.210381 Sep 08 14:05:19.508693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1110/2106] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:19 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:19.618501 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1439bae7-a2e2-494f-ba7d-b60b0ad07d04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:19.619123 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1439bae7-a2e2-494f-ba7d-b60b0ad07d04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:19.625819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1439bae7-a2e2-494f-ba7d-b60b0ad07d04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:19.626308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1439bae7-a2e2-494f-ba7d-b60b0ad07d04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:19.626746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1439bae7-a2e2-494f-ba7d-b60b0ad07d04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:19.627461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1439bae7-a2e2-494f-ba7d-b60b0ad07d04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:19.627914 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1439bae7-a2e2-494f-ba7d-b60b0ad07d04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:19.628340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1439bae7-a2e2-494f-ba7d-b60b0ad07d04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:19.714597 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1439bae7-a2e2-494f-ba7d-b60b0ad07d04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:19.724738 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1439bae7-a2e2-494f-ba7d-b60b0ad07d04 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.110971 Sep 08 14:05:19.725898 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 997/2107] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:19 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:20.200880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-05bdf5c0-53c4-4e86-bcff-1c4dda166c7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:20.203096 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-05bdf5c0-53c4-4e86-bcff-1c4dda166c7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:20.210513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05bdf5c0-53c4-4e86-bcff-1c4dda166c7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:20.211184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05bdf5c0-53c4-4e86-bcff-1c4dda166c7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:20.211696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05bdf5c0-53c4-4e86-bcff-1c4dda166c7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:20.212450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05bdf5c0-53c4-4e86-bcff-1c4dda166c7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:20.212945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05bdf5c0-53c4-4e86-bcff-1c4dda166c7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:20.213495 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05bdf5c0-53c4-4e86-bcff-1c4dda166c7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:20.253864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-05bdf5c0-53c4-4e86-bcff-1c4dda166c7b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057563 Sep 08 14:05:20.254719 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1111/2108] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:20 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:20.263720 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-75c343d0-6b1e-4bcf-936a-6869fb95a0d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:20.266132 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-75c343d0-6b1e-4bcf-936a-6869fb95a0d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:20.274563 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-75c343d0-6b1e-4bcf-936a-6869fb95a0d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:20.275153 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-75c343d0-6b1e-4bcf-936a-6869fb95a0d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:20.275638 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-75c343d0-6b1e-4bcf-936a-6869fb95a0d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:20.276416 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-75c343d0-6b1e-4bcf-936a-6869fb95a0d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:20.276904 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-75c343d0-6b1e-4bcf-936a-6869fb95a0d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:20.277384 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-75c343d0-6b1e-4bcf-936a-6869fb95a0d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:20.310817 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-75c343d0-6b1e-4bcf-936a-6869fb95a0d0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051712 Sep 08 14:05:20.311548 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 998/2109] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:20 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:20.540063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8b94d0cd-de10-41bd-8f7a-ea216adee52d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:20.540628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8b94d0cd-de10-41bd-8f7a-ea216adee52d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:20.548400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b94d0cd-de10-41bd-8f7a-ea216adee52d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:20.548890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b94d0cd-de10-41bd-8f7a-ea216adee52d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:20.549331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b94d0cd-de10-41bd-8f7a-ea216adee52d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:20.550048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b94d0cd-de10-41bd-8f7a-ea216adee52d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:20.550529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b94d0cd-de10-41bd-8f7a-ea216adee52d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:20.550954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b94d0cd-de10-41bd-8f7a-ea216adee52d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:20.725863 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8b94d0cd-de10-41bd-8f7a-ea216adee52d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:20.735374 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8b94d0cd-de10-41bd-8f7a-ea216adee52d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.203969 Sep 08 14:05:20.735949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1112/2110] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:20 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:20.754747 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a78d6e69-12b9-4f5c-ba09-0a2c4baa2d8c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:20.755213 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a78d6e69-12b9-4f5c-ba09-0a2c4baa2d8c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:20.761468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a78d6e69-12b9-4f5c-ba09-0a2c4baa2d8c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:20.761941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a78d6e69-12b9-4f5c-ba09-0a2c4baa2d8c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:20.762400 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a78d6e69-12b9-4f5c-ba09-0a2c4baa2d8c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:20.763090 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a78d6e69-12b9-4f5c-ba09-0a2c4baa2d8c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:20.763544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a78d6e69-12b9-4f5c-ba09-0a2c4baa2d8c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:20.763952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a78d6e69-12b9-4f5c-ba09-0a2c4baa2d8c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:20.841937 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a78d6e69-12b9-4f5c-ba09-0a2c4baa2d8c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:20.852549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a78d6e69-12b9-4f5c-ba09-0a2c4baa2d8c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.101873 Sep 08 14:05:20.853139 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 999/2111] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:20 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:21.267808 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0df2fff4-98cd-4c2e-b4b7-217a5ca75fd0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:21.270209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0df2fff4-98cd-4c2e-b4b7-217a5ca75fd0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:21.277447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0df2fff4-98cd-4c2e-b4b7-217a5ca75fd0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:21.277949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0df2fff4-98cd-4c2e-b4b7-217a5ca75fd0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:21.278427 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0df2fff4-98cd-4c2e-b4b7-217a5ca75fd0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:21.279238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0df2fff4-98cd-4c2e-b4b7-217a5ca75fd0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:21.279698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0df2fff4-98cd-4c2e-b4b7-217a5ca75fd0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:21.280126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0df2fff4-98cd-4c2e-b4b7-217a5ca75fd0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:21.319982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0df2fff4-98cd-4c2e-b4b7-217a5ca75fd0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055883 Sep 08 14:05:21.320961 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1113/2112] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:21 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:21.324964 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8001dd6a-a35f-4f6f-874c-5df4afbe5b4e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:21.327350 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8001dd6a-a35f-4f6f-874c-5df4afbe5b4e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:21.334454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8001dd6a-a35f-4f6f-874c-5df4afbe5b4e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:21.335190 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8001dd6a-a35f-4f6f-874c-5df4afbe5b4e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:21.335767 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8001dd6a-a35f-4f6f-874c-5df4afbe5b4e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:21.336566 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8001dd6a-a35f-4f6f-874c-5df4afbe5b4e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:21.337026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8001dd6a-a35f-4f6f-874c-5df4afbe5b4e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:21.337455 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8001dd6a-a35f-4f6f-874c-5df4afbe5b4e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:21.372750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8001dd6a-a35f-4f6f-874c-5df4afbe5b4e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052552 Sep 08 14:05:21.373358 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1000/2113] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:21 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:21.759968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d071bb5d-878c-4d3f-be3a-0d31491f82c0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:21.760814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d071bb5d-878c-4d3f-be3a-0d31491f82c0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:21.768161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d071bb5d-878c-4d3f-be3a-0d31491f82c0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:21.768708 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d071bb5d-878c-4d3f-be3a-0d31491f82c0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:21.770054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d071bb5d-878c-4d3f-be3a-0d31491f82c0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:21.771003 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d071bb5d-878c-4d3f-be3a-0d31491f82c0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:21.771639 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d071bb5d-878c-4d3f-be3a-0d31491f82c0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:21.772150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d071bb5d-878c-4d3f-be3a-0d31491f82c0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:21.877317 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-aa9791b2-66bd-47b2-b6fb-7337d753e40d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:21.877953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-aa9791b2-66bd-47b2-b6fb-7337d753e40d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:21.884792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa9791b2-66bd-47b2-b6fb-7337d753e40d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:21.885266 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa9791b2-66bd-47b2-b6fb-7337d753e40d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:21.885708 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa9791b2-66bd-47b2-b6fb-7337d753e40d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:21.886481 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa9791b2-66bd-47b2-b6fb-7337d753e40d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:21.886943 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa9791b2-66bd-47b2-b6fb-7337d753e40d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:21.887371 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa9791b2-66bd-47b2-b6fb-7337d753e40d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:21.933966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d071bb5d-878c-4d3f-be3a-0d31491f82c0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:21.942428 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d071bb5d-878c-4d3f-be3a-0d31491f82c0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.187022 Sep 08 14:05:21.943183 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1114/2114] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:21 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:21.968986 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-aa9791b2-66bd-47b2-b6fb-7337d753e40d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:21.977937 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-aa9791b2-66bd-47b2-b6fb-7337d753e40d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.104466 Sep 08 14:05:21.978556 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1001/2115] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:21 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:22.337592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3d0024e3-b5b1-4796-84cb-d4a541ec3518 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:22.341172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3d0024e3-b5b1-4796-84cb-d4a541ec3518 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:22.352801 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d0024e3-b5b1-4796-84cb-d4a541ec3518 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:22.353592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d0024e3-b5b1-4796-84cb-d4a541ec3518 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:22.354345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d0024e3-b5b1-4796-84cb-d4a541ec3518 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:22.355620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d0024e3-b5b1-4796-84cb-d4a541ec3518 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:22.356400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d0024e3-b5b1-4796-84cb-d4a541ec3518 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:22.357170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d0024e3-b5b1-4796-84cb-d4a541ec3518 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:22.387074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-29eec3e9-3720-4eb6-99c3-5cb3eaf71a28 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:22.389254 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-29eec3e9-3720-4eb6-99c3-5cb3eaf71a28 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:22.395959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29eec3e9-3720-4eb6-99c3-5cb3eaf71a28 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:22.396456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29eec3e9-3720-4eb6-99c3-5cb3eaf71a28 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:22.396891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29eec3e9-3720-4eb6-99c3-5cb3eaf71a28 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:22.397732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29eec3e9-3720-4eb6-99c3-5cb3eaf71a28 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:22.398212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29eec3e9-3720-4eb6-99c3-5cb3eaf71a28 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:22.398632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29eec3e9-3720-4eb6-99c3-5cb3eaf71a28 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:22.408814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3d0024e3-b5b1-4796-84cb-d4a541ec3518 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.076416 Sep 08 14:05:22.409387 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1115/2116] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:22 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:22.450461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-29eec3e9-3720-4eb6-99c3-5cb3eaf71a28 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067697 Sep 08 14:05:22.451564 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1002/2117] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:22 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:22.973446 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a91fc737-6792-4935-8010-36b1ae933deb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:22.973949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a91fc737-6792-4935-8010-36b1ae933deb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:22.981315 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a91fc737-6792-4935-8010-36b1ae933deb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:22.981813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a91fc737-6792-4935-8010-36b1ae933deb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:22.982293 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a91fc737-6792-4935-8010-36b1ae933deb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:22.982995 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a91fc737-6792-4935-8010-36b1ae933deb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:22.983456 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a91fc737-6792-4935-8010-36b1ae933deb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:22.983883 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a91fc737-6792-4935-8010-36b1ae933deb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:23.006226 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-96f68d8a-b2c7-4d98-ad1d-786cf37463c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:23.007297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-96f68d8a-b2c7-4d98-ad1d-786cf37463c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:23.017352 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-96f68d8a-b2c7-4d98-ad1d-786cf37463c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:23.017928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-96f68d8a-b2c7-4d98-ad1d-786cf37463c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:23.018679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-96f68d8a-b2c7-4d98-ad1d-786cf37463c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:23.019852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-96f68d8a-b2c7-4d98-ad1d-786cf37463c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:23.020681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-96f68d8a-b2c7-4d98-ad1d-786cf37463c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:23.021396 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-96f68d8a-b2c7-4d98-ad1d-786cf37463c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:23.104889 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-96f68d8a-b2c7-4d98-ad1d-786cf37463c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:23.113095 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-96f68d8a-b2c7-4d98-ad1d-786cf37463c7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.113751 Sep 08 14:05:23.113849 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1003/2118] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:22 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:23.185172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a91fc737-6792-4935-8010-36b1ae933deb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:23.197688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a91fc737-6792-4935-8010-36b1ae933deb tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.227049 Sep 08 14:05:23.198421 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1116/2119] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:22 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:23.427276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8041ead2-3bc5-46d6-8454-ca5a94925d41 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:23.430398 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8041ead2-3bc5-46d6-8454-ca5a94925d41 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:23.437153 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8041ead2-3bc5-46d6-8454-ca5a94925d41 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:23.437631 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8041ead2-3bc5-46d6-8454-ca5a94925d41 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:23.438061 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8041ead2-3bc5-46d6-8454-ca5a94925d41 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:23.438839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8041ead2-3bc5-46d6-8454-ca5a94925d41 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:23.439237 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8041ead2-3bc5-46d6-8454-ca5a94925d41 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:23.439654 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8041ead2-3bc5-46d6-8454-ca5a94925d41 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:23.465761 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9401b57b-5a6a-48a4-b40b-ec6f2ca1dd16 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:23.468032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9401b57b-5a6a-48a4-b40b-ec6f2ca1dd16 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:23.475118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9401b57b-5a6a-48a4-b40b-ec6f2ca1dd16 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:23.475815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9401b57b-5a6a-48a4-b40b-ec6f2ca1dd16 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:23.476250 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9401b57b-5a6a-48a4-b40b-ec6f2ca1dd16 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:23.477079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9401b57b-5a6a-48a4-b40b-ec6f2ca1dd16 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:23.477745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9401b57b-5a6a-48a4-b40b-ec6f2ca1dd16 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:23.478197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9401b57b-5a6a-48a4-b40b-ec6f2ca1dd16 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:23.479639 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8041ead2-3bc5-46d6-8454-ca5a94925d41 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059665 Sep 08 14:05:23.480430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1004/2120] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:23 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:23.522056 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9401b57b-5a6a-48a4-b40b-ec6f2ca1dd16 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060688 Sep 08 14:05:23.522677 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1117/2121] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:23 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:24.136800 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4222bc66-c67e-413d-b25d-8634027156f2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:24.137409 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4222bc66-c67e-413d-b25d-8634027156f2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:24.143603 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4222bc66-c67e-413d-b25d-8634027156f2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:24.144125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4222bc66-c67e-413d-b25d-8634027156f2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:24.144559 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4222bc66-c67e-413d-b25d-8634027156f2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:24.145273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4222bc66-c67e-413d-b25d-8634027156f2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:24.145722 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4222bc66-c67e-413d-b25d-8634027156f2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:24.146161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4222bc66-c67e-413d-b25d-8634027156f2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:24.171481 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:24.222736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2fea2432-2eb2-46cb-916a-39f5faf9b6be tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:24.223627 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2fea2432-2eb2-46cb-916a-39f5faf9b6be tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:24.224585 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4222bc66-c67e-413d-b25d-8634027156f2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:24.231094 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fea2432-2eb2-46cb-916a-39f5faf9b6be tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:24.231593 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fea2432-2eb2-46cb-916a-39f5faf9b6be tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:24.232034 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fea2432-2eb2-46cb-916a-39f5faf9b6be tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:24.232852 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fea2432-2eb2-46cb-916a-39f5faf9b6be tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:24.233584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fea2432-2eb2-46cb-916a-39f5faf9b6be tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:24.233750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4222bc66-c67e-413d-b25d-8634027156f2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.100752 Sep 08 14:05:24.234343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1005/2122] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:24 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:24.234498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2fea2432-2eb2-46cb-916a-39f5faf9b6be tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:24.408979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2fea2432-2eb2-46cb-916a-39f5faf9b6be tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:24.419272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2fea2432-2eb2-46cb-916a-39f5faf9b6be tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.200804 Sep 08 14:05:24.419861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1118/2123] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:24 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:24.496928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4b83afc2-8ca1-4840-b214-d8601d23e1b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:24.500069 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4b83afc2-8ca1-4840-b214-d8601d23e1b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:24.511522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b83afc2-8ca1-4840-b214-d8601d23e1b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:24.512385 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b83afc2-8ca1-4840-b214-d8601d23e1b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:24.512988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b83afc2-8ca1-4840-b214-d8601d23e1b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:24.514207 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b83afc2-8ca1-4840-b214-d8601d23e1b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:24.514809 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b83afc2-8ca1-4840-b214-d8601d23e1b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:24.515402 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4b83afc2-8ca1-4840-b214-d8601d23e1b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:24.535584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0149eea6-e65d-4552-82ba-625445bbd9ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:24.538479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0149eea6-e65d-4552-82ba-625445bbd9ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:24.546086 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0149eea6-e65d-4552-82ba-625445bbd9ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:24.546624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0149eea6-e65d-4552-82ba-625445bbd9ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:24.547071 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0149eea6-e65d-4552-82ba-625445bbd9ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:24.547882 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0149eea6-e65d-4552-82ba-625445bbd9ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:24.548393 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0149eea6-e65d-4552-82ba-625445bbd9ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:24.548923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0149eea6-e65d-4552-82ba-625445bbd9ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:24.558220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4b83afc2-8ca1-4840-b214-d8601d23e1b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.068109 Sep 08 14:05:24.559283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1006/2124] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:24 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:24.585783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0149eea6-e65d-4552-82ba-625445bbd9ee tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054314 Sep 08 14:05:24.586488 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1119/2125] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:24 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:25.264492 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-691aebc4-0262-48db-ab34-d3660c0956a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:25.265107 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-691aebc4-0262-48db-ab34-d3660c0956a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:25.271645 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-691aebc4-0262-48db-ab34-d3660c0956a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:25.272237 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-691aebc4-0262-48db-ab34-d3660c0956a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:25.272676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-691aebc4-0262-48db-ab34-d3660c0956a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:25.273391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-691aebc4-0262-48db-ab34-d3660c0956a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:25.273873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-691aebc4-0262-48db-ab34-d3660c0956a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:25.274336 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-691aebc4-0262-48db-ab34-d3660c0956a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:25.343965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-691aebc4-0262-48db-ab34-d3660c0956a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:25.352457 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-691aebc4-0262-48db-ab34-d3660c0956a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.092028 Sep 08 14:05:25.353146 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1007/2126] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:25 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:25.447593 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-afc871c9-8421-4bad-8182-96eaf08c63e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:25.448200 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-afc871c9-8421-4bad-8182-96eaf08c63e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:25.454282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-afc871c9-8421-4bad-8182-96eaf08c63e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:25.454770 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-afc871c9-8421-4bad-8182-96eaf08c63e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:25.455210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-afc871c9-8421-4bad-8182-96eaf08c63e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:25.455898 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-afc871c9-8421-4bad-8182-96eaf08c63e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:25.456350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-afc871c9-8421-4bad-8182-96eaf08c63e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:25.456774 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-afc871c9-8421-4bad-8182-96eaf08c63e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:25.573727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-28864f16-5f2d-41dd-98db-022e382b52ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:25.576341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-28864f16-5f2d-41dd-98db-022e382b52ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:25.589523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-28864f16-5f2d-41dd-98db-022e382b52ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:25.590018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-28864f16-5f2d-41dd-98db-022e382b52ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:25.590497 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-28864f16-5f2d-41dd-98db-022e382b52ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:25.591330 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-28864f16-5f2d-41dd-98db-022e382b52ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:25.593070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-28864f16-5f2d-41dd-98db-022e382b52ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:25.593070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-28864f16-5f2d-41dd-98db-022e382b52ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:25.605869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-afc871c9-8421-4bad-8182-96eaf08c63e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:25.617025 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-afc871c9-8421-4bad-8182-96eaf08c63e6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.170910 Sep 08 14:05:25.617025 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1120/2127] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:25 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:25.621153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8b15f7bb-f694-4111-b2f8-3f01e65c3f6c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:25.624080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8b15f7bb-f694-4111-b2f8-3f01e65c3f6c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:25.635351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-28864f16-5f2d-41dd-98db-022e382b52ee tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063715 Sep 08 14:05:25.635660 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b15f7bb-f694-4111-b2f8-3f01e65c3f6c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:25.635660 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b15f7bb-f694-4111-b2f8-3f01e65c3f6c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:25.635660 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b15f7bb-f694-4111-b2f8-3f01e65c3f6c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:25.636143 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b15f7bb-f694-4111-b2f8-3f01e65c3f6c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:25.636293 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b15f7bb-f694-4111-b2f8-3f01e65c3f6c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:25.636711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b15f7bb-f694-4111-b2f8-3f01e65c3f6c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:25.639319 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1008/2128] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:25 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:25.670084 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8b15f7bb-f694-4111-b2f8-3f01e65c3f6c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052823 Sep 08 14:05:25.670681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1121/2129] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:25 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:26.377070 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-67e1e9cb-492c-4e21-9614-67e771dc6a32 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:26.377777 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-67e1e9cb-492c-4e21-9614-67e771dc6a32 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:26.385122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67e1e9cb-492c-4e21-9614-67e771dc6a32 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:26.385849 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67e1e9cb-492c-4e21-9614-67e771dc6a32 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:26.386393 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67e1e9cb-492c-4e21-9614-67e771dc6a32 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:26.387203 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67e1e9cb-492c-4e21-9614-67e771dc6a32 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:26.387780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67e1e9cb-492c-4e21-9614-67e771dc6a32 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:26.388270 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-67e1e9cb-492c-4e21-9614-67e771dc6a32 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:26.466336 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-67e1e9cb-492c-4e21-9614-67e771dc6a32 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:26.474580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-67e1e9cb-492c-4e21-9614-67e771dc6a32 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.101381 Sep 08 14:05:26.475176 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1009/2130] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:26 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:26.646756 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d7c31a74-f0f9-46e9-b236-2fc217812a49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:26.647461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9cb49570-04e5-4853-9000-60030ea5b360 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:26.647866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d7c31a74-f0f9-46e9-b236-2fc217812a49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:26.649239 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9cb49570-04e5-4853-9000-60030ea5b360 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:26.654438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d7c31a74-f0f9-46e9-b236-2fc217812a49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:26.654920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d7c31a74-f0f9-46e9-b236-2fc217812a49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:26.655355 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d7c31a74-f0f9-46e9-b236-2fc217812a49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:26.656092 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9cb49570-04e5-4853-9000-60030ea5b360 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:26.656231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d7c31a74-f0f9-46e9-b236-2fc217812a49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:26.656572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d7c31a74-f0f9-46e9-b236-2fc217812a49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:26.656694 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9cb49570-04e5-4853-9000-60030ea5b360 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:26.656988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d7c31a74-f0f9-46e9-b236-2fc217812a49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:26.657114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9cb49570-04e5-4853-9000-60030ea5b360 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:26.657690 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9cb49570-04e5-4853-9000-60030ea5b360 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:26.658171 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9cb49570-04e5-4853-9000-60030ea5b360 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:26.658604 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9cb49570-04e5-4853-9000-60030ea5b360 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:26.697375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9cb49570-04e5-4853-9000-60030ea5b360 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053982 Sep 08 14:05:26.698905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1010/2131] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:26 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:26.703710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-77b3f619-d2b3-41de-9a4d-567b802c7523 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:26.705925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-77b3f619-d2b3-41de-9a4d-567b802c7523 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:26.712052 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77b3f619-d2b3-41de-9a4d-567b802c7523 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:26.712529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77b3f619-d2b3-41de-9a4d-567b802c7523 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:26.712953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77b3f619-d2b3-41de-9a4d-567b802c7523 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:26.713641 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77b3f619-d2b3-41de-9a4d-567b802c7523 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:26.714089 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77b3f619-d2b3-41de-9a4d-567b802c7523 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:26.714528 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-77b3f619-d2b3-41de-9a4d-567b802c7523 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:26.747671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-77b3f619-d2b3-41de-9a4d-567b802c7523 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047993 Sep 08 14:05:26.748251 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1011/2132] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:26 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:26.808627 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d7c31a74-f0f9-46e9-b236-2fc217812a49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:26.817048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d7c31a74-f0f9-46e9-b236-2fc217812a49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.174237 Sep 08 14:05:26.817645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1122/2133] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:26 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:27.498563 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b216a232-1c07-4be4-bcbe-334daf8b0feb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:27.505495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b216a232-1c07-4be4-bcbe-334daf8b0feb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:27.505495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:05:27.505495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:05:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 (ip 10.222.0.25) !!! Sep 08 14:05:27.515767 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b216a232-1c07-4be4-bcbe-334daf8b0feb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:27.516261 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b216a232-1c07-4be4-bcbe-334daf8b0feb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:27.516828 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b216a232-1c07-4be4-bcbe-334daf8b0feb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:27.517417 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b216a232-1c07-4be4-bcbe-334daf8b0feb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:27.517890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b216a232-1c07-4be4-bcbe-334daf8b0feb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:27.518557 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b216a232-1c07-4be4-bcbe-334daf8b0feb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:27.611112 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b216a232-1c07-4be4-bcbe-334daf8b0feb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:27.620149 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b216a232-1c07-4be4-bcbe-334daf8b0feb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.125506 Sep 08 14:05:27.620762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1012/2134] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:27 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:27.714411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f28a652c-36ae-413a-b162-e2be9f22611f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:27.718180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f28a652c-36ae-413a-b162-e2be9f22611f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:27.730076 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f28a652c-36ae-413a-b162-e2be9f22611f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:27.731033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f28a652c-36ae-413a-b162-e2be9f22611f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:27.731835 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f28a652c-36ae-413a-b162-e2be9f22611f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:27.733145 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f28a652c-36ae-413a-b162-e2be9f22611f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:27.733978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f28a652c-36ae-413a-b162-e2be9f22611f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:27.734782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f28a652c-36ae-413a-b162-e2be9f22611f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:27.761602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-59eedfa8-633e-4d8c-bb54-0fdc33cc6625 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:27.763845 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-59eedfa8-633e-4d8c-bb54-0fdc33cc6625 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:27.771215 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59eedfa8-633e-4d8c-bb54-0fdc33cc6625 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:27.772293 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59eedfa8-633e-4d8c-bb54-0fdc33cc6625 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:27.774717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59eedfa8-633e-4d8c-bb54-0fdc33cc6625 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:27.774717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59eedfa8-633e-4d8c-bb54-0fdc33cc6625 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:27.774717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59eedfa8-633e-4d8c-bb54-0fdc33cc6625 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:27.774717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59eedfa8-633e-4d8c-bb54-0fdc33cc6625 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:27.779258 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f28a652c-36ae-413a-b162-e2be9f22611f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.072014 Sep 08 14:05:27.780231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1123/2135] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:27 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:27.810119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-59eedfa8-633e-4d8c-bb54-0fdc33cc6625 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052830 Sep 08 14:05:27.814992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1013/2136] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:27 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:27.842589 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2f8e39f0-7def-414a-b9a1-644049c8287a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:27.843192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2f8e39f0-7def-414a-b9a1-644049c8287a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:27.850339 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f8e39f0-7def-414a-b9a1-644049c8287a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:27.850838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f8e39f0-7def-414a-b9a1-644049c8287a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:27.851272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f8e39f0-7def-414a-b9a1-644049c8287a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:27.851998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f8e39f0-7def-414a-b9a1-644049c8287a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:27.852449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f8e39f0-7def-414a-b9a1-644049c8287a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:27.852869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2f8e39f0-7def-414a-b9a1-644049c8287a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:28.024953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2f8e39f0-7def-414a-b9a1-644049c8287a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:28.034498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2f8e39f0-7def-414a-b9a1-644049c8287a tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.196417 Sep 08 14:05:28.035117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1124/2137] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:27 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:28.644975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-845f65df-4a99-4d15-8e3b-166a32480cad tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:28.645597 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-845f65df-4a99-4d15-8e3b-166a32480cad tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:28.653194 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-845f65df-4a99-4d15-8e3b-166a32480cad tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:28.653706 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-845f65df-4a99-4d15-8e3b-166a32480cad tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:28.654163 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-845f65df-4a99-4d15-8e3b-166a32480cad tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:28.654900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-845f65df-4a99-4d15-8e3b-166a32480cad tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:28.655353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-845f65df-4a99-4d15-8e3b-166a32480cad tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:28.655781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-845f65df-4a99-4d15-8e3b-166a32480cad tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:28.722976 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-845f65df-4a99-4d15-8e3b-166a32480cad tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:28.733133 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-845f65df-4a99-4d15-8e3b-166a32480cad tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.092002 Sep 08 14:05:28.733764 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1014/2138] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:28 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:28.794990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f882940a-2224-408c-b3a4-579e871ca4bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:28.797141 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f882940a-2224-408c-b3a4-579e871ca4bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:28.803613 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f882940a-2224-408c-b3a4-579e871ca4bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:28.804104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f882940a-2224-408c-b3a4-579e871ca4bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:28.804542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f882940a-2224-408c-b3a4-579e871ca4bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:28.805230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f882940a-2224-408c-b3a4-579e871ca4bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:28.805682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f882940a-2224-408c-b3a4-579e871ca4bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:28.806167 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f882940a-2224-408c-b3a4-579e871ca4bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:28.827803 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7c2f59f4-4c7e-4379-9ac4-3bf1a63ed437 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:28.829946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7c2f59f4-4c7e-4379-9ac4-3bf1a63ed437 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:28.837353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c2f59f4-4c7e-4379-9ac4-3bf1a63ed437 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:28.837843 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c2f59f4-4c7e-4379-9ac4-3bf1a63ed437 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:28.838313 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c2f59f4-4c7e-4379-9ac4-3bf1a63ed437 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:28.838504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f882940a-2224-408c-b3a4-579e871ca4bc tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047380 Sep 08 14:05:28.839253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c2f59f4-4c7e-4379-9ac4-3bf1a63ed437 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:28.839369 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1125/2139] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:28 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:28.839708 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c2f59f4-4c7e-4379-9ac4-3bf1a63ed437 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:28.840130 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c2f59f4-4c7e-4379-9ac4-3bf1a63ed437 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:28.872994 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7c2f59f4-4c7e-4379-9ac4-3bf1a63ed437 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049026 Sep 08 14:05:28.873655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1015/2140] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:28 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:29.057561 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9739d857-50bf-487d-a132-4475da5c9e0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:29.058244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9739d857-50bf-487d-a132-4475da5c9e0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:29.064471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9739d857-50bf-487d-a132-4475da5c9e0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:29.064951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9739d857-50bf-487d-a132-4475da5c9e0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:29.065478 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9739d857-50bf-487d-a132-4475da5c9e0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:29.066081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9739d857-50bf-487d-a132-4475da5c9e0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:29.066580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9739d857-50bf-487d-a132-4475da5c9e0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:29.066981 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9739d857-50bf-487d-a132-4475da5c9e0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:29.082017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:29.202503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9739d857-50bf-487d-a132-4475da5c9e0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:29.211678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9739d857-50bf-487d-a132-4475da5c9e0f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.158020 Sep 08 14:05:29.212261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1126/2141] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:29 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:29.756909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3ed9e9b8-4a36-4168-bcea-905513b8fe75 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:29.758121 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3ed9e9b8-4a36-4168-bcea-905513b8fe75 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:29.760201 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:29.764883 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3ed9e9b8-4a36-4168-bcea-905513b8fe75 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:29.765359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3ed9e9b8-4a36-4168-bcea-905513b8fe75 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:29.765835 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3ed9e9b8-4a36-4168-bcea-905513b8fe75 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:29.766526 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3ed9e9b8-4a36-4168-bcea-905513b8fe75 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:29.782186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3ed9e9b8-4a36-4168-bcea-905513b8fe75 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:29.782186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3ed9e9b8-4a36-4168-bcea-905513b8fe75 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:29.831250 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3ed9e9b8-4a36-4168-bcea-905513b8fe75 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:29.839715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3ed9e9b8-4a36-4168-bcea-905513b8fe75 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.086683 Sep 08 14:05:29.840298 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1016/2142] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:29 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:29.851863 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a0d4c4ad-a880-4136-baca-0ffaabae9f3a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:29.854177 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a0d4c4ad-a880-4136-baca-0ffaabae9f3a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:29.860873 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a0d4c4ad-a880-4136-baca-0ffaabae9f3a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:29.861357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a0d4c4ad-a880-4136-baca-0ffaabae9f3a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:29.861795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a0d4c4ad-a880-4136-baca-0ffaabae9f3a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:29.862534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a0d4c4ad-a880-4136-baca-0ffaabae9f3a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:29.862966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a0d4c4ad-a880-4136-baca-0ffaabae9f3a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:29.863390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a0d4c4ad-a880-4136-baca-0ffaabae9f3a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:29.885331 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5d2dff56-060f-4b8f-afc8-4707884f4e08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:29.887608 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5d2dff56-060f-4b8f-afc8-4707884f4e08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:29.894307 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d2dff56-060f-4b8f-afc8-4707884f4e08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:29.895771 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d2dff56-060f-4b8f-afc8-4707884f4e08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:29.896231 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d2dff56-060f-4b8f-afc8-4707884f4e08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:29.897247 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a0d4c4ad-a880-4136-baca-0ffaabae9f3a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050045 Sep 08 14:05:29.902169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1127/2143] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:29 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:29.902346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d2dff56-060f-4b8f-afc8-4707884f4e08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:29.902346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d2dff56-060f-4b8f-afc8-4707884f4e08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:29.902346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d2dff56-060f-4b8f-afc8-4707884f4e08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:29.932961 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5d2dff56-060f-4b8f-afc8-4707884f4e08 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051702 Sep 08 14:05:29.933559 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1017/2144] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:29 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:30.236712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-30d34e08-4647-4918-909b-0d955201ee51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:30.237226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-30d34e08-4647-4918-909b-0d955201ee51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:30.243672 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-30d34e08-4647-4918-909b-0d955201ee51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:30.244152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-30d34e08-4647-4918-909b-0d955201ee51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:30.244581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-30d34e08-4647-4918-909b-0d955201ee51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:30.245256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-30d34e08-4647-4918-909b-0d955201ee51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:30.245702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-30d34e08-4647-4918-909b-0d955201ee51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:30.246121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-30d34e08-4647-4918-909b-0d955201ee51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:30.382427 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-30d34e08-4647-4918-909b-0d955201ee51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:30.390602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-30d34e08-4647-4918-909b-0d955201ee51 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.158116 Sep 08 14:05:30.391179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1128/2145] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:30 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:30.863839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cfb5ca26-3faf-4b91-be16-f248712d76bb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:30.864507 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cfb5ca26-3faf-4b91-be16-f248712d76bb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:30.871519 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cfb5ca26-3faf-4b91-be16-f248712d76bb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:30.872003 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cfb5ca26-3faf-4b91-be16-f248712d76bb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:30.872435 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cfb5ca26-3faf-4b91-be16-f248712d76bb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:30.873169 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cfb5ca26-3faf-4b91-be16-f248712d76bb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:30.873642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cfb5ca26-3faf-4b91-be16-f248712d76bb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:30.874127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cfb5ca26-3faf-4b91-be16-f248712d76bb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:30.909051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-dcfd6c3e-3e32-4541-90e3-9b9eaaf8ed2d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:30.911432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-dcfd6c3e-3e32-4541-90e3-9b9eaaf8ed2d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:30.913562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:05:30.913764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:05:30 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments (ip 10.222.0.25) !!! Sep 08 14:05:30.920496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dcfd6c3e-3e32-4541-90e3-9b9eaaf8ed2d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:30.921204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dcfd6c3e-3e32-4541-90e3-9b9eaaf8ed2d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:30.921722 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dcfd6c3e-3e32-4541-90e3-9b9eaaf8ed2d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:30.922522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dcfd6c3e-3e32-4541-90e3-9b9eaaf8ed2d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:30.923051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dcfd6c3e-3e32-4541-90e3-9b9eaaf8ed2d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:30.923549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dcfd6c3e-3e32-4541-90e3-9b9eaaf8ed2d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:30.946088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-cfb5ca26-3faf-4b91-be16-f248712d76bb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:30.954251 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cfb5ca26-3faf-4b91-be16-f248712d76bb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.094371 Sep 08 14:05:30.954838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1018/2146] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:30 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:30.958034 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-dcfd6c3e-3e32-4541-90e3-9b9eaaf8ed2d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052877 Sep 08 14:05:30.958946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1129/2147] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:30 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:30.959648 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f6fd5b61-7595-4c27-b757-3b0bc619c438 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:30.962422 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f6fd5b61-7595-4c27-b757-3b0bc619c438 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:30.968945 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f6fd5b61-7595-4c27-b757-3b0bc619c438 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:30.969432 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f6fd5b61-7595-4c27-b757-3b0bc619c438 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:30.969874 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f6fd5b61-7595-4c27-b757-3b0bc619c438 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:30.970627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f6fd5b61-7595-4c27-b757-3b0bc619c438 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:30.971090 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f6fd5b61-7595-4c27-b757-3b0bc619c438 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:30.971514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f6fd5b61-7595-4c27-b757-3b0bc619c438 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:31.004214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f6fd5b61-7595-4c27-b757-3b0bc619c438 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048229 Sep 08 14:05:31.004799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1019/2148] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:30 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:31.415179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-763153a3-6f95-4bbd-9683-59f7b1aab86f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:31.415709 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-763153a3-6f95-4bbd-9683-59f7b1aab86f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:31.421796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-763153a3-6f95-4bbd-9683-59f7b1aab86f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:31.422338 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-763153a3-6f95-4bbd-9683-59f7b1aab86f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:31.422860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-763153a3-6f95-4bbd-9683-59f7b1aab86f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:31.423545 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-763153a3-6f95-4bbd-9683-59f7b1aab86f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:31.424001 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-763153a3-6f95-4bbd-9683-59f7b1aab86f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:31.424429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-763153a3-6f95-4bbd-9683-59f7b1aab86f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:31.562231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-763153a3-6f95-4bbd-9683-59f7b1aab86f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:31.570371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-763153a3-6f95-4bbd-9683-59f7b1aab86f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.159552 Sep 08 14:05:31.570942 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1130/2149] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:31 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:31.971381 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-40f55ab4-c4a5-4e6d-9fbf-e7e768375acd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:31.973544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-40f55ab4-c4a5-4e6d-9fbf-e7e768375acd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:31.981015 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ae177ebc-70a2-4c70-9ca3-484bb3f04c6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:31.981637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ae177ebc-70a2-4c70-9ca3-484bb3f04c6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:31.981768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-40f55ab4-c4a5-4e6d-9fbf-e7e768375acd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:31.982327 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-40f55ab4-c4a5-4e6d-9fbf-e7e768375acd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:31.982859 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-40f55ab4-c4a5-4e6d-9fbf-e7e768375acd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:31.983530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-40f55ab4-c4a5-4e6d-9fbf-e7e768375acd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:31.984001 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-40f55ab4-c4a5-4e6d-9fbf-e7e768375acd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:31.984953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-40f55ab4-c4a5-4e6d-9fbf-e7e768375acd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:31.988510 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae177ebc-70a2-4c70-9ca3-484bb3f04c6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:31.999681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae177ebc-70a2-4c70-9ca3-484bb3f04c6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:31.999681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae177ebc-70a2-4c70-9ca3-484bb3f04c6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:31.999681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae177ebc-70a2-4c70-9ca3-484bb3f04c6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:31.999681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae177ebc-70a2-4c70-9ca3-484bb3f04c6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:32.000059 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae177ebc-70a2-4c70-9ca3-484bb3f04c6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:32.023807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-40f55ab4-c4a5-4e6d-9fbf-e7e768375acd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056452 Sep 08 14:05:32.024387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1020/2150] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:31 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:32.029010 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-dc6584d5-436c-41fb-bd38-cf4eb2e6323f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:32.031172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-dc6584d5-436c-41fb-bd38-cf4eb2e6323f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:32.037397 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dc6584d5-436c-41fb-bd38-cf4eb2e6323f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:32.037892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dc6584d5-436c-41fb-bd38-cf4eb2e6323f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:32.044460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dc6584d5-436c-41fb-bd38-cf4eb2e6323f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:32.044460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dc6584d5-436c-41fb-bd38-cf4eb2e6323f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:32.044460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dc6584d5-436c-41fb-bd38-cf4eb2e6323f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:32.044460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-dc6584d5-436c-41fb-bd38-cf4eb2e6323f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:32.058495 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ae177ebc-70a2-4c70-9ca3-484bb3f04c6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:32.066650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ae177ebc-70a2-4c70-9ca3-484bb3f04c6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.092212 Sep 08 14:05:32.067143 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1131/2151] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:31 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:32.071624 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-dc6584d5-436c-41fb-bd38-cf4eb2e6323f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046203 Sep 08 14:05:32.072226 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1021/2152] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:32 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:32.592986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-56e20503-d354-486c-b6b1-17e2a891f3d3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:32.593597 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-56e20503-d354-486c-b6b1-17e2a891f3d3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:32.601209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56e20503-d354-486c-b6b1-17e2a891f3d3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:32.602061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56e20503-d354-486c-b6b1-17e2a891f3d3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:32.602703 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56e20503-d354-486c-b6b1-17e2a891f3d3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:32.603450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56e20503-d354-486c-b6b1-17e2a891f3d3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:32.603909 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56e20503-d354-486c-b6b1-17e2a891f3d3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:32.604488 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56e20503-d354-486c-b6b1-17e2a891f3d3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:32.752443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-56e20503-d354-486c-b6b1-17e2a891f3d3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:32.767044 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-56e20503-d354-486c-b6b1-17e2a891f3d3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.177524 Sep 08 14:05:32.768109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1132/2153] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:32 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:33.036618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-468fc225-d7e1-4889-ae52-e3ee2cdeab34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:33.039195 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-468fc225-d7e1-4889-ae52-e3ee2cdeab34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:33.045663 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-468fc225-d7e1-4889-ae52-e3ee2cdeab34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:33.046202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-468fc225-d7e1-4889-ae52-e3ee2cdeab34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:33.046915 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-468fc225-d7e1-4889-ae52-e3ee2cdeab34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:33.047322 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-468fc225-d7e1-4889-ae52-e3ee2cdeab34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:33.047788 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-468fc225-d7e1-4889-ae52-e3ee2cdeab34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:33.048237 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-468fc225-d7e1-4889-ae52-e3ee2cdeab34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:33.051993 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:33.084755 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-468fc225-d7e1-4889-ae52-e3ee2cdeab34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052179 Sep 08 14:05:33.085779 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1022/2154] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:33 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:33.087729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ebea1e29-d30b-4c83-aacb-059b6cfa6c82 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:33.090408 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ebea1e29-d30b-4c83-aacb-059b6cfa6c82 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:33.091513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6acd42ae-7e11-4aff-a384-e074c974963c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:33.092478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6acd42ae-7e11-4aff-a384-e074c974963c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:33.098282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ebea1e29-d30b-4c83-aacb-059b6cfa6c82 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:33.102391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ebea1e29-d30b-4c83-aacb-059b6cfa6c82 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:33.102391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ebea1e29-d30b-4c83-aacb-059b6cfa6c82 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:33.102391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ebea1e29-d30b-4c83-aacb-059b6cfa6c82 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:33.102391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ebea1e29-d30b-4c83-aacb-059b6cfa6c82 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:33.103062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ebea1e29-d30b-4c83-aacb-059b6cfa6c82 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:33.103498 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6acd42ae-7e11-4aff-a384-e074c974963c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:33.104218 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6acd42ae-7e11-4aff-a384-e074c974963c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:33.104927 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6acd42ae-7e11-4aff-a384-e074c974963c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:33.105809 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6acd42ae-7e11-4aff-a384-e074c974963c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:33.108410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6acd42ae-7e11-4aff-a384-e074c974963c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:33.108410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6acd42ae-7e11-4aff-a384-e074c974963c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:33.145681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ebea1e29-d30b-4c83-aacb-059b6cfa6c82 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064084 Sep 08 14:05:33.146296 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1133/2155] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:33 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:33.192080 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6acd42ae-7e11-4aff-a384-e074c974963c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:33.202883 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6acd42ae-7e11-4aff-a384-e074c974963c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.116153 Sep 08 14:05:33.203473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1023/2156] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:33 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:33.793051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a7f08fa0-a492-40f7-a151-a0f908fb36a9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:33.793662 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a7f08fa0-a492-40f7-a151-a0f908fb36a9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:33.800810 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7f08fa0-a492-40f7-a151-a0f908fb36a9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:33.801304 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7f08fa0-a492-40f7-a151-a0f908fb36a9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:33.801741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7f08fa0-a492-40f7-a151-a0f908fb36a9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:33.802495 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7f08fa0-a492-40f7-a151-a0f908fb36a9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:33.802957 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7f08fa0-a492-40f7-a151-a0f908fb36a9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:33.803388 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7f08fa0-a492-40f7-a151-a0f908fb36a9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:33.956679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a7f08fa0-a492-40f7-a151-a0f908fb36a9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:33.967043 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a7f08fa0-a492-40f7-a151-a0f908fb36a9 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.177833 Sep 08 14:05:33.967642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1134/2157] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:33 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:34.099415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-31c17b31-4d8f-46ed-afbb-25b48cf86edb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:34.102215 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-31c17b31-4d8f-46ed-afbb-25b48cf86edb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:34.109027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31c17b31-4d8f-46ed-afbb-25b48cf86edb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:34.109512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31c17b31-4d8f-46ed-afbb-25b48cf86edb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:34.109974 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31c17b31-4d8f-46ed-afbb-25b48cf86edb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:34.110739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31c17b31-4d8f-46ed-afbb-25b48cf86edb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:34.111164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31c17b31-4d8f-46ed-afbb-25b48cf86edb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:34.111580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31c17b31-4d8f-46ed-afbb-25b48cf86edb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:34.147614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-31c17b31-4d8f-46ed-afbb-25b48cf86edb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052426 Sep 08 14:05:34.148189 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1024/2158] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:34 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:34.159443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e917ad17-fce8-4293-98fb-97d6eab4e15b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:34.161662 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e917ad17-fce8-4293-98fb-97d6eab4e15b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:34.172431 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e917ad17-fce8-4293-98fb-97d6eab4e15b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:34.173331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e917ad17-fce8-4293-98fb-97d6eab4e15b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:34.174121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e917ad17-fce8-4293-98fb-97d6eab4e15b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:34.174970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e917ad17-fce8-4293-98fb-97d6eab4e15b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:34.175497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e917ad17-fce8-4293-98fb-97d6eab4e15b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:34.175929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e917ad17-fce8-4293-98fb-97d6eab4e15b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:34.207987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e917ad17-fce8-4293-98fb-97d6eab4e15b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052661 Sep 08 14:05:34.208556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1135/2159] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:34 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:34.228884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c45603f2-a978-498c-99d4-4276b3890b30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:34.229493 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c45603f2-a978-498c-99d4-4276b3890b30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:34.235687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c45603f2-a978-498c-99d4-4276b3890b30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:34.236238 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c45603f2-a978-498c-99d4-4276b3890b30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:34.236662 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c45603f2-a978-498c-99d4-4276b3890b30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:34.237522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c45603f2-a978-498c-99d4-4276b3890b30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:34.238021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c45603f2-a978-498c-99d4-4276b3890b30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:34.238450 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c45603f2-a978-498c-99d4-4276b3890b30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:34.306115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c45603f2-a978-498c-99d4-4276b3890b30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:34.314307 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c45603f2-a978-498c-99d4-4276b3890b30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.089260 Sep 08 14:05:34.314872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1025/2160] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:34 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:34.991475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-91af4e7f-fbdc-4ce6-9ce6-07fdc15f2dde tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:34.991998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-91af4e7f-fbdc-4ce6-9ce6-07fdc15f2dde tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:34.998649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91af4e7f-fbdc-4ce6-9ce6-07fdc15f2dde tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:34.999138 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91af4e7f-fbdc-4ce6-9ce6-07fdc15f2dde tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:34.999581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91af4e7f-fbdc-4ce6-9ce6-07fdc15f2dde tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:35.000287 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91af4e7f-fbdc-4ce6-9ce6-07fdc15f2dde tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:35.000743 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91af4e7f-fbdc-4ce6-9ce6-07fdc15f2dde tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:35.001174 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91af4e7f-fbdc-4ce6-9ce6-07fdc15f2dde tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:35.158637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-91af4e7f-fbdc-4ce6-9ce6-07fdc15f2dde tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:35.160861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6fcd64be-1983-489d-8624-e0793007e3ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:35.163073 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6fcd64be-1983-489d-8624-e0793007e3ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:35.169941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6fcd64be-1983-489d-8624-e0793007e3ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:35.170462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6fcd64be-1983-489d-8624-e0793007e3ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:35.170891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6fcd64be-1983-489d-8624-e0793007e3ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:35.171298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-91af4e7f-fbdc-4ce6-9ce6-07fdc15f2dde tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.184278 Sep 08 14:05:35.172035 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6fcd64be-1983-489d-8624-e0793007e3ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:35.172035 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6fcd64be-1983-489d-8624-e0793007e3ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:35.172407 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6fcd64be-1983-489d-8624-e0793007e3ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:35.174591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1136/2161] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:34 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:35.206580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6fcd64be-1983-489d-8624-e0793007e3ba tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049768 Sep 08 14:05:35.207160 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1026/2162] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:35 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:35.220583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-416a2dca-1fcd-42e0-91fe-b356cfe13989 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:35.222780 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-416a2dca-1fcd-42e0-91fe-b356cfe13989 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:35.229708 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-416a2dca-1fcd-42e0-91fe-b356cfe13989 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:35.230220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-416a2dca-1fcd-42e0-91fe-b356cfe13989 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:35.230656 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-416a2dca-1fcd-42e0-91fe-b356cfe13989 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:35.231516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-416a2dca-1fcd-42e0-91fe-b356cfe13989 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:35.231984 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-416a2dca-1fcd-42e0-91fe-b356cfe13989 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:35.232404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-416a2dca-1fcd-42e0-91fe-b356cfe13989 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:35.263380 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-416a2dca-1fcd-42e0-91fe-b356cfe13989 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046641 Sep 08 14:05:35.263868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1137/2163] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:35 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:35.337156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bc27c2ab-bdee-4419-95b9-7760aea8864b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:35.337783 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bc27c2ab-bdee-4419-95b9-7760aea8864b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:35.344531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc27c2ab-bdee-4419-95b9-7760aea8864b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:35.345019 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc27c2ab-bdee-4419-95b9-7760aea8864b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:35.345458 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc27c2ab-bdee-4419-95b9-7760aea8864b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:35.346196 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc27c2ab-bdee-4419-95b9-7760aea8864b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:35.346992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc27c2ab-bdee-4419-95b9-7760aea8864b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:35.347548 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bc27c2ab-bdee-4419-95b9-7760aea8864b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:35.419646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-bc27c2ab-bdee-4419-95b9-7760aea8864b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:35.427674 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bc27c2ab-bdee-4419-95b9-7760aea8864b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.094434 Sep 08 14:05:35.428252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1027/2164] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:35 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:36.197357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e6bc8281-bfd2-4866-8e1d-5d40b7596ce2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:36.197991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e6bc8281-bfd2-4866-8e1d-5d40b7596ce2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:36.204529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6bc8281-bfd2-4866-8e1d-5d40b7596ce2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:36.205015 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6bc8281-bfd2-4866-8e1d-5d40b7596ce2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:36.205445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6bc8281-bfd2-4866-8e1d-5d40b7596ce2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:36.206130 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6bc8281-bfd2-4866-8e1d-5d40b7596ce2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:36.206601 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6bc8281-bfd2-4866-8e1d-5d40b7596ce2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:36.207021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6bc8281-bfd2-4866-8e1d-5d40b7596ce2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:36.219459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bac176b0-4885-468a-94c5-b9b16cd88d34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:36.221797 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bac176b0-4885-468a-94c5-b9b16cd88d34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:36.228940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bac176b0-4885-468a-94c5-b9b16cd88d34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:36.229460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bac176b0-4885-468a-94c5-b9b16cd88d34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:36.229898 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bac176b0-4885-468a-94c5-b9b16cd88d34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:36.230660 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bac176b0-4885-468a-94c5-b9b16cd88d34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:36.231119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bac176b0-4885-468a-94c5-b9b16cd88d34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:36.231554 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bac176b0-4885-468a-94c5-b9b16cd88d34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:36.269135 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bac176b0-4885-468a-94c5-b9b16cd88d34 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053416 Sep 08 14:05:36.270236 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1028/2165] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:36 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:36.281256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-eea1790b-7e7f-4a74-8b0b-9f8f3aef5bb1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:36.283751 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-eea1790b-7e7f-4a74-8b0b-9f8f3aef5bb1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:36.290102 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eea1790b-7e7f-4a74-8b0b-9f8f3aef5bb1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:36.290634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eea1790b-7e7f-4a74-8b0b-9f8f3aef5bb1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:36.291063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eea1790b-7e7f-4a74-8b0b-9f8f3aef5bb1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:36.291773 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eea1790b-7e7f-4a74-8b0b-9f8f3aef5bb1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:36.292312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eea1790b-7e7f-4a74-8b0b-9f8f3aef5bb1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:36.292738 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eea1790b-7e7f-4a74-8b0b-9f8f3aef5bb1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:36.331493 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-eea1790b-7e7f-4a74-8b0b-9f8f3aef5bb1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054076 Sep 08 14:05:36.332914 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1029/2166] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:36 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:36.381358 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e6bc8281-bfd2-4866-8e1d-5d40b7596ce2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:36.396565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e6bc8281-bfd2-4866-8e1d-5d40b7596ce2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.203773 Sep 08 14:05:36.397153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1138/2167] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:36 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:36.452244 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-605692b1-833c-46d2-8fd3-4734e8fb37de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:36.452879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-605692b1-833c-46d2-8fd3-4734e8fb37de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:36.459424 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-605692b1-833c-46d2-8fd3-4734e8fb37de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:36.459910 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-605692b1-833c-46d2-8fd3-4734e8fb37de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:36.460342 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-605692b1-833c-46d2-8fd3-4734e8fb37de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:36.461142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-605692b1-833c-46d2-8fd3-4734e8fb37de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:36.461591 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-605692b1-833c-46d2-8fd3-4734e8fb37de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:36.462016 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-605692b1-833c-46d2-8fd3-4734e8fb37de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:36.531715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-605692b1-833c-46d2-8fd3-4734e8fb37de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:36.540436 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-605692b1-833c-46d2-8fd3-4734e8fb37de tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.092242 Sep 08 14:05:36.541029 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1030/2168] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:36 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:37.285152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-76b68fb4-fba7-4936-a17c-2feddd615490 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:37.288208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-76b68fb4-fba7-4936-a17c-2feddd615490 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:37.301628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-76b68fb4-fba7-4936-a17c-2feddd615490 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:37.302165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-76b68fb4-fba7-4936-a17c-2feddd615490 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:37.302603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-76b68fb4-fba7-4936-a17c-2feddd615490 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:37.303329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-76b68fb4-fba7-4936-a17c-2feddd615490 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:37.303800 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-76b68fb4-fba7-4936-a17c-2feddd615490 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:37.304230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-76b68fb4-fba7-4936-a17c-2feddd615490 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:37.340393 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-76b68fb4-fba7-4936-a17c-2feddd615490 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062783 Sep 08 14:05:37.340962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1139/2169] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:37 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:37.349184 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6b8940d5-907c-462a-863b-f8ea39013ca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:37.352702 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6b8940d5-907c-462a-863b-f8ea39013ca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:37.363707 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6b8940d5-907c-462a-863b-f8ea39013ca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:37.364280 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6b8940d5-907c-462a-863b-f8ea39013ca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:37.364751 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6b8940d5-907c-462a-863b-f8ea39013ca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:37.366241 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6b8940d5-907c-462a-863b-f8ea39013ca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:37.367161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6b8940d5-907c-462a-863b-f8ea39013ca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:37.367678 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6b8940d5-907c-462a-863b-f8ea39013ca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:37.410676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6b8940d5-907c-462a-863b-f8ea39013ca0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069171 Sep 08 14:05:37.411255 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1031/2170] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:37 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:37.421217 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3f078013-1fb4-4b75-9d9d-44c84f814244 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:37.421858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3f078013-1fb4-4b75-9d9d-44c84f814244 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:37.425116 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:37.443389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3f078013-1fb4-4b75-9d9d-44c84f814244 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:37.443950 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3f078013-1fb4-4b75-9d9d-44c84f814244 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:37.444705 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3f078013-1fb4-4b75-9d9d-44c84f814244 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:37.446543 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3f078013-1fb4-4b75-9d9d-44c84f814244 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:37.447538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3f078013-1fb4-4b75-9d9d-44c84f814244 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:37.448192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3f078013-1fb4-4b75-9d9d-44c84f814244 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:37.567713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d4e31e9e-6cf0-4f9d-a945-41b8955f31fe tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:37.568339 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d4e31e9e-6cf0-4f9d-a945-41b8955f31fe tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:37.575747 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d4e31e9e-6cf0-4f9d-a945-41b8955f31fe tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:37.576742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d4e31e9e-6cf0-4f9d-a945-41b8955f31fe tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:37.577360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d4e31e9e-6cf0-4f9d-a945-41b8955f31fe tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:37.578121 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d4e31e9e-6cf0-4f9d-a945-41b8955f31fe tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:37.578121 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d4e31e9e-6cf0-4f9d-a945-41b8955f31fe tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:37.578592 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d4e31e9e-6cf0-4f9d-a945-41b8955f31fe tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:37.638322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3f078013-1fb4-4b75-9d9d-44c84f814244 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:37.647838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3f078013-1fb4-4b75-9d9d-44c84f814244 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.231032 Sep 08 14:05:37.657672 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1140/2171] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:37 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:37.665481 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d4e31e9e-6cf0-4f9d-a945-41b8955f31fe tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:37.675542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d4e31e9e-6cf0-4f9d-a945-41b8955f31fe tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.113484 Sep 08 14:05:37.676486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1032/2172] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:37 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:38.358581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9a7975bf-6e8f-4a57-b8e2-36cc85dfa8bd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:38.360830 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9a7975bf-6e8f-4a57-b8e2-36cc85dfa8bd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:38.368715 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a7975bf-6e8f-4a57-b8e2-36cc85dfa8bd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:38.369215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a7975bf-6e8f-4a57-b8e2-36cc85dfa8bd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:38.369657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a7975bf-6e8f-4a57-b8e2-36cc85dfa8bd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:38.370404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a7975bf-6e8f-4a57-b8e2-36cc85dfa8bd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:38.370867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a7975bf-6e8f-4a57-b8e2-36cc85dfa8bd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:38.371334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a7975bf-6e8f-4a57-b8e2-36cc85dfa8bd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:38.404982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9a7975bf-6e8f-4a57-b8e2-36cc85dfa8bd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051015 Sep 08 14:05:38.405897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1141/2173] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:38 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:38.429219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8478a634-a9d4-4482-8e33-15742b15727d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:38.432003 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8478a634-a9d4-4482-8e33-15742b15727d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:38.439356 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8478a634-a9d4-4482-8e33-15742b15727d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:38.439933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8478a634-a9d4-4482-8e33-15742b15727d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:38.440475 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8478a634-a9d4-4482-8e33-15742b15727d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:38.441252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8478a634-a9d4-4482-8e33-15742b15727d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:38.441769 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8478a634-a9d4-4482-8e33-15742b15727d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:38.442304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8478a634-a9d4-4482-8e33-15742b15727d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:38.477394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8478a634-a9d4-4482-8e33-15742b15727d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052758 Sep 08 14:05:38.478007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1033/2174] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:38 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:38.672228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f57be314-0680-4197-b1af-b45be20b99e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:38.672830 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f57be314-0680-4197-b1af-b45be20b99e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:38.679929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f57be314-0680-4197-b1af-b45be20b99e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:38.680723 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f57be314-0680-4197-b1af-b45be20b99e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:38.681474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f57be314-0680-4197-b1af-b45be20b99e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:38.682720 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f57be314-0680-4197-b1af-b45be20b99e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:38.683287 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f57be314-0680-4197-b1af-b45be20b99e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:38.683727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f57be314-0680-4197-b1af-b45be20b99e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:38.700637 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ff7a7a04-9b64-4dca-a185-e903182fd73d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:38.701306 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ff7a7a04-9b64-4dca-a185-e903182fd73d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:38.708572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff7a7a04-9b64-4dca-a185-e903182fd73d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:38.709068 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff7a7a04-9b64-4dca-a185-e903182fd73d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:38.709505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff7a7a04-9b64-4dca-a185-e903182fd73d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:38.710267 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff7a7a04-9b64-4dca-a185-e903182fd73d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:38.711241 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff7a7a04-9b64-4dca-a185-e903182fd73d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:38.711676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff7a7a04-9b64-4dca-a185-e903182fd73d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:38.784124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ff7a7a04-9b64-4dca-a185-e903182fd73d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:38.799772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ff7a7a04-9b64-4dca-a185-e903182fd73d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.102885 Sep 08 14:05:38.801195 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1034/2175] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:38 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:38.859555 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f57be314-0680-4197-b1af-b45be20b99e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:38.873612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f57be314-0680-4197-b1af-b45be20b99e2 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.205115 Sep 08 14:05:38.874230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1142/2176] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:38 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:39.419766 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-57ea2321-12d0-4794-bbb8-23af23345377 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:39.422183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-57ea2321-12d0-4794-bbb8-23af23345377 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:39.428813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-57ea2321-12d0-4794-bbb8-23af23345377 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:39.429868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-57ea2321-12d0-4794-bbb8-23af23345377 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:39.430477 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-57ea2321-12d0-4794-bbb8-23af23345377 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:39.431175 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-57ea2321-12d0-4794-bbb8-23af23345377 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:39.431625 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-57ea2321-12d0-4794-bbb8-23af23345377 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:39.432048 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-57ea2321-12d0-4794-bbb8-23af23345377 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:39.476142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-57ea2321-12d0-4794-bbb8-23af23345377 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.062454 Sep 08 14:05:39.476728 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1035/2177] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:39 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:39.490373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5c1312ec-df39-4b6b-a8e0-5cba0870d0c5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:39.493373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5c1312ec-df39-4b6b-a8e0-5cba0870d0c5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:39.501661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5c1312ec-df39-4b6b-a8e0-5cba0870d0c5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:39.501661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5c1312ec-df39-4b6b-a8e0-5cba0870d0c5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:39.501661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5c1312ec-df39-4b6b-a8e0-5cba0870d0c5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:39.502208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5c1312ec-df39-4b6b-a8e0-5cba0870d0c5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:39.502647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5c1312ec-df39-4b6b-a8e0-5cba0870d0c5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:39.503075 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5c1312ec-df39-4b6b-a8e0-5cba0870d0c5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:39.505814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:39.541599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5c1312ec-df39-4b6b-a8e0-5cba0870d0c5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055291 Sep 08 14:05:39.542202 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1143/2178] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:39 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:39.836406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7e2fedfb-9eaa-4fdc-96e6-4066301d88bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:39.837017 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7e2fedfb-9eaa-4fdc-96e6-4066301d88bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:39.843231 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e2fedfb-9eaa-4fdc-96e6-4066301d88bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:39.843710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e2fedfb-9eaa-4fdc-96e6-4066301d88bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:39.844216 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e2fedfb-9eaa-4fdc-96e6-4066301d88bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:39.844943 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e2fedfb-9eaa-4fdc-96e6-4066301d88bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:39.845261 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e2fedfb-9eaa-4fdc-96e6-4066301d88bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:39.845675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7e2fedfb-9eaa-4fdc-96e6-4066301d88bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:39.902449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-92c24e0b-7597-4c02-afde-53ca37b5d31f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:39.903078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-92c24e0b-7597-4c02-afde-53ca37b5d31f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:39.909546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-92c24e0b-7597-4c02-afde-53ca37b5d31f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:39.915524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-92c24e0b-7597-4c02-afde-53ca37b5d31f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:39.915524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-92c24e0b-7597-4c02-afde-53ca37b5d31f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:39.915524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-92c24e0b-7597-4c02-afde-53ca37b5d31f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:39.915524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-92c24e0b-7597-4c02-afde-53ca37b5d31f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:39.916299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-92c24e0b-7597-4c02-afde-53ca37b5d31f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:39.921228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7e2fedfb-9eaa-4fdc-96e6-4066301d88bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:39.932397 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7e2fedfb-9eaa-4fdc-96e6-4066301d88bc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.099650 Sep 08 14:05:39.933123 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1036/2179] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:39 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:40.077193 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-92c24e0b-7597-4c02-afde-53ca37b5d31f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:40.086680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-92c24e0b-7597-4c02-afde-53ca37b5d31f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.188422 Sep 08 14:05:40.087195 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1144/2180] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:39 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:40.492764 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a4652318-cfbb-4f29-9bf6-827f6eb27b93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:40.494994 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a4652318-cfbb-4f29-9bf6-827f6eb27b93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:40.502895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a4652318-cfbb-4f29-9bf6-827f6eb27b93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:40.503665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a4652318-cfbb-4f29-9bf6-827f6eb27b93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:40.504146 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a4652318-cfbb-4f29-9bf6-827f6eb27b93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:40.504950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a4652318-cfbb-4f29-9bf6-827f6eb27b93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:40.505447 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a4652318-cfbb-4f29-9bf6-827f6eb27b93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:40.505965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a4652318-cfbb-4f29-9bf6-827f6eb27b93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:40.548108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a4652318-cfbb-4f29-9bf6-827f6eb27b93 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061135 Sep 08 14:05:40.555298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9a5bd63d-53df-48b1-9254-efafc8bcf0d4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:40.555900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1037/2181] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:40 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:40.559074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9a5bd63d-53df-48b1-9254-efafc8bcf0d4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:40.568670 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a5bd63d-53df-48b1-9254-efafc8bcf0d4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:40.569349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a5bd63d-53df-48b1-9254-efafc8bcf0d4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:40.570534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a5bd63d-53df-48b1-9254-efafc8bcf0d4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:40.571927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a5bd63d-53df-48b1-9254-efafc8bcf0d4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:40.572423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a5bd63d-53df-48b1-9254-efafc8bcf0d4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:40.573115 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9a5bd63d-53df-48b1-9254-efafc8bcf0d4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:40.612599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9a5bd63d-53df-48b1-9254-efafc8bcf0d4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061418 Sep 08 14:05:40.613320 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1145/2182] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:40 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:40.957809 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ed826cb6-7edd-42d3-a166-b865c50e163c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:40.958454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ed826cb6-7edd-42d3-a166-b865c50e163c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:40.972885 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed826cb6-7edd-42d3-a166-b865c50e163c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:40.973417 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed826cb6-7edd-42d3-a166-b865c50e163c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:40.973870 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed826cb6-7edd-42d3-a166-b865c50e163c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:40.974645 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed826cb6-7edd-42d3-a166-b865c50e163c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:40.975091 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed826cb6-7edd-42d3-a166-b865c50e163c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:40.975514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed826cb6-7edd-42d3-a166-b865c50e163c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:41.059466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ed826cb6-7edd-42d3-a166-b865c50e163c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:41.063214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:41.067804 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ed826cb6-7edd-42d3-a166-b865c50e163c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.114531 Sep 08 14:05:41.068567 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1038/2183] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:40 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:41.119622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cbdda3a5-096b-4366-9207-9696e39d974d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:41.120501 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cbdda3a5-096b-4366-9207-9696e39d974d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:41.127124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbdda3a5-096b-4366-9207-9696e39d974d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:41.127646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbdda3a5-096b-4366-9207-9696e39d974d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:41.128056 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbdda3a5-096b-4366-9207-9696e39d974d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:41.128760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbdda3a5-096b-4366-9207-9696e39d974d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:41.129215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbdda3a5-096b-4366-9207-9696e39d974d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:41.129644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbdda3a5-096b-4366-9207-9696e39d974d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:41.301577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cbdda3a5-096b-4366-9207-9696e39d974d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:41.311468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cbdda3a5-096b-4366-9207-9696e39d974d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.197800 Sep 08 14:05:41.312353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1146/2184] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:41 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:41.563759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4a2f1177-32ee-44fb-bcb9-609536e0629e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:41.565955 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4a2f1177-32ee-44fb-bcb9-609536e0629e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:41.573155 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a2f1177-32ee-44fb-bcb9-609536e0629e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:41.573633 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a2f1177-32ee-44fb-bcb9-609536e0629e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:41.574065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a2f1177-32ee-44fb-bcb9-609536e0629e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:41.574785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a2f1177-32ee-44fb-bcb9-609536e0629e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:41.575232 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a2f1177-32ee-44fb-bcb9-609536e0629e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:41.575649 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4a2f1177-32ee-44fb-bcb9-609536e0629e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:41.617059 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4a2f1177-32ee-44fb-bcb9-609536e0629e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057206 Sep 08 14:05:41.617654 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1039/2185] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:41 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:41.630568 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-19e7c87e-ecbe-4076-b425-37d0bbffc5ef tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:41.633234 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-19e7c87e-ecbe-4076-b425-37d0bbffc5ef tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:41.643176 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-19e7c87e-ecbe-4076-b425-37d0bbffc5ef tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:41.643969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-19e7c87e-ecbe-4076-b425-37d0bbffc5ef tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:41.644721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-19e7c87e-ecbe-4076-b425-37d0bbffc5ef tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:41.645959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-19e7c87e-ecbe-4076-b425-37d0bbffc5ef tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:41.646792 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-19e7c87e-ecbe-4076-b425-37d0bbffc5ef tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:41.647553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-19e7c87e-ecbe-4076-b425-37d0bbffc5ef tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:41.689880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-19e7c87e-ecbe-4076-b425-37d0bbffc5ef tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064567 Sep 08 14:05:41.691755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1147/2186] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:41 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:42.097873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-015e3b9f-136e-477e-86cc-d44b68fec303 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:42.098691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-015e3b9f-136e-477e-86cc-d44b68fec303 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:42.106114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-015e3b9f-136e-477e-86cc-d44b68fec303 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:42.106685 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-015e3b9f-136e-477e-86cc-d44b68fec303 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:42.107160 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-015e3b9f-136e-477e-86cc-d44b68fec303 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:42.107878 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-015e3b9f-136e-477e-86cc-d44b68fec303 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:42.108330 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-015e3b9f-136e-477e-86cc-d44b68fec303 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:42.108765 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-015e3b9f-136e-477e-86cc-d44b68fec303 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:42.239043 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-015e3b9f-136e-477e-86cc-d44b68fec303 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:42.259510 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-015e3b9f-136e-477e-86cc-d44b68fec303 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.168241 Sep 08 14:05:42.260589 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1040/2187] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:42 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:42.340127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-26b7de69-0c09-4159-8e6a-1a629b633ba0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:42.340749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-26b7de69-0c09-4159-8e6a-1a629b633ba0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:42.348921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26b7de69-0c09-4159-8e6a-1a629b633ba0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:42.349786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26b7de69-0c09-4159-8e6a-1a629b633ba0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:42.350610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26b7de69-0c09-4159-8e6a-1a629b633ba0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:42.351765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26b7de69-0c09-4159-8e6a-1a629b633ba0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:42.352587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26b7de69-0c09-4159-8e6a-1a629b633ba0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:42.353380 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26b7de69-0c09-4159-8e6a-1a629b633ba0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:42.525705 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-26b7de69-0c09-4159-8e6a-1a629b633ba0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:42.534434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-26b7de69-0c09-4159-8e6a-1a629b633ba0 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.201873 Sep 08 14:05:42.535580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1148/2188] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:42 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:42.630921 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2d5cf3e7-bf6f-4a19-9c1b-f6023077b9f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:42.633552 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2d5cf3e7-bf6f-4a19-9c1b-f6023077b9f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:42.641332 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2d5cf3e7-bf6f-4a19-9c1b-f6023077b9f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:42.642033 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2d5cf3e7-bf6f-4a19-9c1b-f6023077b9f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:42.642738 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2d5cf3e7-bf6f-4a19-9c1b-f6023077b9f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:42.643633 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2d5cf3e7-bf6f-4a19-9c1b-f6023077b9f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:42.644255 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2d5cf3e7-bf6f-4a19-9c1b-f6023077b9f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:42.644800 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2d5cf3e7-bf6f-4a19-9c1b-f6023077b9f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:42.683373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2d5cf3e7-bf6f-4a19-9c1b-f6023077b9f1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057161 Sep 08 14:05:42.684001 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1041/2189] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:42 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:42.706261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7017486a-322e-48fd-82dd-205a165a0c5c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:42.709794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7017486a-322e-48fd-82dd-205a165a0c5c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:42.717257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7017486a-322e-48fd-82dd-205a165a0c5c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:42.717908 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7017486a-322e-48fd-82dd-205a165a0c5c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:42.718392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7017486a-322e-48fd-82dd-205a165a0c5c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:42.719121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7017486a-322e-48fd-82dd-205a165a0c5c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:42.719584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7017486a-322e-48fd-82dd-205a165a0c5c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:42.720007 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7017486a-322e-48fd-82dd-205a165a0c5c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:42.758515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7017486a-322e-48fd-82dd-205a165a0c5c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059027 Sep 08 14:05:42.759093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1149/2190] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:42 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:43.298118 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c520fc03-733f-4786-a862-b0bfb8b227df tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:43.299325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c520fc03-733f-4786-a862-b0bfb8b227df tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:43.306798 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c520fc03-733f-4786-a862-b0bfb8b227df tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:43.307285 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c520fc03-733f-4786-a862-b0bfb8b227df tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:43.307723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c520fc03-733f-4786-a862-b0bfb8b227df tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:43.308465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c520fc03-733f-4786-a862-b0bfb8b227df tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:43.309023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c520fc03-733f-4786-a862-b0bfb8b227df tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:43.309456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c520fc03-733f-4786-a862-b0bfb8b227df tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:43.405779 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c520fc03-733f-4786-a862-b0bfb8b227df tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:43.420113 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c520fc03-733f-4786-a862-b0bfb8b227df tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.127198 Sep 08 14:05:43.420716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1042/2191] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:43 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:43.568607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8a4ca6a4-694e-4e45-91c1-d577d65f0d08 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:43.569142 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8a4ca6a4-694e-4e45-91c1-d577d65f0d08 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:43.576831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a4ca6a4-694e-4e45-91c1-d577d65f0d08 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:43.577508 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a4ca6a4-694e-4e45-91c1-d577d65f0d08 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:43.577941 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a4ca6a4-694e-4e45-91c1-d577d65f0d08 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:43.578681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a4ca6a4-694e-4e45-91c1-d577d65f0d08 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:43.579134 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a4ca6a4-694e-4e45-91c1-d577d65f0d08 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:43.579556 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a4ca6a4-694e-4e45-91c1-d577d65f0d08 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:43.700210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-841515cd-1116-46e2-b4e5-f9a99df12ef4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:43.703052 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-841515cd-1116-46e2-b4e5-f9a99df12ef4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:43.711876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-841515cd-1116-46e2-b4e5-f9a99df12ef4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:43.712371 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-841515cd-1116-46e2-b4e5-f9a99df12ef4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:43.712804 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-841515cd-1116-46e2-b4e5-f9a99df12ef4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:43.713965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-841515cd-1116-46e2-b4e5-f9a99df12ef4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:43.714455 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-841515cd-1116-46e2-b4e5-f9a99df12ef4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:43.714880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-841515cd-1116-46e2-b4e5-f9a99df12ef4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:43.773989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-841515cd-1116-46e2-b4e5-f9a99df12ef4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.079641 Sep 08 14:05:43.775759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1043/2192] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:43 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:43.779365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8a4ca6a4-694e-4e45-91c1-d577d65f0d08 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:43.782700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a6b757b5-cc06-4f3d-8271-84cdb3277c42 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:43.785021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a6b757b5-cc06-4f3d-8271-84cdb3277c42 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:43.789996 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8a4ca6a4-694e-4e45-91c1-d577d65f0d08 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.227881 Sep 08 14:05:43.790736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1150/2193] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:43 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:43.792820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6b757b5-cc06-4f3d-8271-84cdb3277c42 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:43.793318 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6b757b5-cc06-4f3d-8271-84cdb3277c42 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:43.793901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6b757b5-cc06-4f3d-8271-84cdb3277c42 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:43.794479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6b757b5-cc06-4f3d-8271-84cdb3277c42 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:43.794909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6b757b5-cc06-4f3d-8271-84cdb3277c42 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:43.795332 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6b757b5-cc06-4f3d-8271-84cdb3277c42 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:43.832533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a6b757b5-cc06-4f3d-8271-84cdb3277c42 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055532 Sep 08 14:05:43.832998 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1044/2194] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:43 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:44.446734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fb81a391-4a48-461d-ab2a-5dd7a73c2458 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:44.447348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fb81a391-4a48-461d-ab2a-5dd7a73c2458 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:44.455447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb81a391-4a48-461d-ab2a-5dd7a73c2458 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:44.463279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb81a391-4a48-461d-ab2a-5dd7a73c2458 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:44.463279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb81a391-4a48-461d-ab2a-5dd7a73c2458 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:44.463279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb81a391-4a48-461d-ab2a-5dd7a73c2458 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:44.463279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb81a391-4a48-461d-ab2a-5dd7a73c2458 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:44.463751 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb81a391-4a48-461d-ab2a-5dd7a73c2458 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:44.537698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fb81a391-4a48-461d-ab2a-5dd7a73c2458 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:44.557145 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fb81a391-4a48-461d-ab2a-5dd7a73c2458 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.114301 Sep 08 14:05:44.557743 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1151/2195] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:44 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:44.791541 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-11aed56f-139a-4ad3-8013-f8a6415e0c4d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:44.793479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-11aed56f-139a-4ad3-8013-f8a6415e0c4d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:44.800778 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-11aed56f-139a-4ad3-8013-f8a6415e0c4d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:44.801267 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-11aed56f-139a-4ad3-8013-f8a6415e0c4d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:44.801699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-11aed56f-139a-4ad3-8013-f8a6415e0c4d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:44.802565 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-11aed56f-139a-4ad3-8013-f8a6415e0c4d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:44.803052 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-11aed56f-139a-4ad3-8013-f8a6415e0c4d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:44.806798 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-11aed56f-139a-4ad3-8013-f8a6415e0c4d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:44.816418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-70943649-6801-4c9c-a351-4f5d978fb996 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:44.818064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-70943649-6801-4c9c-a351-4f5d978fb996 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:44.827228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70943649-6801-4c9c-a351-4f5d978fb996 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:44.827228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70943649-6801-4c9c-a351-4f5d978fb996 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:44.827666 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70943649-6801-4c9c-a351-4f5d978fb996 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:44.828803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70943649-6801-4c9c-a351-4f5d978fb996 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:44.829623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70943649-6801-4c9c-a351-4f5d978fb996 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:44.830762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70943649-6801-4c9c-a351-4f5d978fb996 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:44.845836 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-11aed56f-139a-4ad3-8013-f8a6415e0c4d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058267 Sep 08 14:05:44.852849 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1045/2196] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:44 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:44.858109 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-11f11877-5962-455b-b176-9622ddbc17a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:44.860561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-11f11877-5962-455b-b176-9622ddbc17a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:44.875106 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-11f11877-5962-455b-b176-9622ddbc17a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:44.875662 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-11f11877-5962-455b-b176-9622ddbc17a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:44.876125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-11f11877-5962-455b-b176-9622ddbc17a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:44.876819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-11f11877-5962-455b-b176-9622ddbc17a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:44.877277 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-11f11877-5962-455b-b176-9622ddbc17a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:44.877807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-11f11877-5962-455b-b176-9622ddbc17a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:44.927456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-11f11877-5962-455b-b176-9622ddbc17a3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.073523 Sep 08 14:05:44.928083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1046/2197] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:44 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:45.021881 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-70943649-6801-4c9c-a351-4f5d978fb996 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:45.031102 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-70943649-6801-4c9c-a351-4f5d978fb996 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.220707 Sep 08 14:05:45.031686 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1152/2198] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:44 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:45.134412 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-55e3f370-f4b9-4387-93a9-8453ffc55833 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:45.135006 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-55e3f370-f4b9-4387-93a9-8453ffc55833 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:45.142503 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-55e3f370-f4b9-4387-93a9-8453ffc55833 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:45.143176 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-55e3f370-f4b9-4387-93a9-8453ffc55833 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:45.143916 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-55e3f370-f4b9-4387-93a9-8453ffc55833 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:45.144792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-55e3f370-f4b9-4387-93a9-8453ffc55833 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:45.145413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-55e3f370-f4b9-4387-93a9-8453ffc55833 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:45.145962 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-55e3f370-f4b9-4387-93a9-8453ffc55833 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:45.582766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-432c0d3b-25d5-4ef0-9166-88b67eca7ac6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:45.583218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-432c0d3b-25d5-4ef0-9166-88b67eca7ac6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:45.590025 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-432c0d3b-25d5-4ef0-9166-88b67eca7ac6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:45.590565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-432c0d3b-25d5-4ef0-9166-88b67eca7ac6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:45.591043 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-432c0d3b-25d5-4ef0-9166-88b67eca7ac6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:45.591756 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-432c0d3b-25d5-4ef0-9166-88b67eca7ac6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:45.592215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-432c0d3b-25d5-4ef0-9166-88b67eca7ac6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:45.592649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-432c0d3b-25d5-4ef0-9166-88b67eca7ac6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:45.676915 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-432c0d3b-25d5-4ef0-9166-88b67eca7ac6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:45.685210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-432c0d3b-25d5-4ef0-9166-88b67eca7ac6 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.107214 Sep 08 14:05:45.685790 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1153/2199] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:45 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:45.860508 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2e58f6c7-8755-471f-9b73-7d308923eed7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:45.862726 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2e58f6c7-8755-471f-9b73-7d308923eed7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:45.870001 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e58f6c7-8755-471f-9b73-7d308923eed7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:45.870531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e58f6c7-8755-471f-9b73-7d308923eed7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:45.870977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e58f6c7-8755-471f-9b73-7d308923eed7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:45.871680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e58f6c7-8755-471f-9b73-7d308923eed7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:45.872165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e58f6c7-8755-471f-9b73-7d308923eed7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:45.872602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e58f6c7-8755-471f-9b73-7d308923eed7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:46.052945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2e58f6c7-8755-471f-9b73-7d308923eed7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.197269 Sep 08 14:05:46.053504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1154/2200] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:45 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:46.058602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1539dda0-c074-421a-9bdc-d6348b4dd568 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:46.060688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1539dda0-c074-421a-9bdc-d6348b4dd568 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:46.066477 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1539dda0-c074-421a-9bdc-d6348b4dd568 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:46.066964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1539dda0-c074-421a-9bdc-d6348b4dd568 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:46.067400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1539dda0-c074-421a-9bdc-d6348b4dd568 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:46.068089 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1539dda0-c074-421a-9bdc-d6348b4dd568 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:46.068542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1539dda0-c074-421a-9bdc-d6348b4dd568 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:46.068967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1539dda0-c074-421a-9bdc-d6348b4dd568 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:46.100587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1539dda0-c074-421a-9bdc-d6348b4dd568 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045474 Sep 08 14:05:46.101194 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1155/2201] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:46 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:46.105549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bc1c6769-3762-4730-b88d-f5471425e64f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:46.106161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bc1c6769-3762-4730-b88d-f5471425e64f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:46.120772 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bc1c6769-3762-4730-b88d-f5471425e64f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:46.120772 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bc1c6769-3762-4730-b88d-f5471425e64f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:46.120772 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bc1c6769-3762-4730-b88d-f5471425e64f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:46.120772 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bc1c6769-3762-4730-b88d-f5471425e64f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:46.121396 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bc1c6769-3762-4730-b88d-f5471425e64f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:46.121396 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bc1c6769-3762-4730-b88d-f5471425e64f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:46.255248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bc1c6769-3762-4730-b88d-f5471425e64f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:46.264043 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bc1c6769-3762-4730-b88d-f5471425e64f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.161868 Sep 08 14:05:46.264610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1156/2202] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:46 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:46.708790 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f391701b-c98a-4a63-bf46-c264374d7214 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:46.709411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f391701b-c98a-4a63-bf46-c264374d7214 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:46.716114 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f391701b-c98a-4a63-bf46-c264374d7214 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:46.716652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f391701b-c98a-4a63-bf46-c264374d7214 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:46.717044 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f391701b-c98a-4a63-bf46-c264374d7214 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:46.717748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f391701b-c98a-4a63-bf46-c264374d7214 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:46.718359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f391701b-c98a-4a63-bf46-c264374d7214 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:46.718694 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f391701b-c98a-4a63-bf46-c264374d7214 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:46.783661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f391701b-c98a-4a63-bf46-c264374d7214 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:46.792547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f391701b-c98a-4a63-bf46-c264374d7214 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.087830 Sep 08 14:05:46.793118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1157/2203] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:46 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:47.066477 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4234f3c3-1b0d-4fe8-a521-6b0bbd08d7a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:47.068608 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4234f3c3-1b0d-4fe8-a521-6b0bbd08d7a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:47.075737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4234f3c3-1b0d-4fe8-a521-6b0bbd08d7a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:47.076232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4234f3c3-1b0d-4fe8-a521-6b0bbd08d7a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:47.076674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4234f3c3-1b0d-4fe8-a521-6b0bbd08d7a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:47.077380 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4234f3c3-1b0d-4fe8-a521-6b0bbd08d7a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:47.077837 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4234f3c3-1b0d-4fe8-a521-6b0bbd08d7a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:47.078311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4234f3c3-1b0d-4fe8-a521-6b0bbd08d7a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:47.109889 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4234f3c3-1b0d-4fe8-a521-6b0bbd08d7a4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047591 Sep 08 14:05:47.110479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1158/2204] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:47 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:47.127475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1540150f-d730-47b3-a88a-66f96bfcccb7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:47.129599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1540150f-d730-47b3-a88a-66f96bfcccb7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:47.136256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1540150f-d730-47b3-a88a-66f96bfcccb7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:47.136749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1540150f-d730-47b3-a88a-66f96bfcccb7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:47.137187 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1540150f-d730-47b3-a88a-66f96bfcccb7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:47.137940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1540150f-d730-47b3-a88a-66f96bfcccb7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:47.138430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1540150f-d730-47b3-a88a-66f96bfcccb7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:47.138862 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1540150f-d730-47b3-a88a-66f96bfcccb7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:47.164824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:47.171331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1540150f-d730-47b3-a88a-66f96bfcccb7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047744 Sep 08 14:05:47.178224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1159/2205] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:47 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:47.289360 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fae41b20-44a9-48fc-a424-12c0313c9e9b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:47.290059 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fae41b20-44a9-48fc-a424-12c0313c9e9b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:47.296539 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fae41b20-44a9-48fc-a424-12c0313c9e9b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:47.297021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fae41b20-44a9-48fc-a424-12c0313c9e9b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:47.297455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fae41b20-44a9-48fc-a424-12c0313c9e9b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:47.298159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fae41b20-44a9-48fc-a424-12c0313c9e9b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:47.298624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fae41b20-44a9-48fc-a424-12c0313c9e9b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:47.299089 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fae41b20-44a9-48fc-a424-12c0313c9e9b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:47.473172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fae41b20-44a9-48fc-a424-12c0313c9e9b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:47.481666 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fae41b20-44a9-48fc-a424-12c0313c9e9b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.197224 Sep 08 14:05:47.482647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1160/2206] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:47 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:47.816268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-56423749-a179-446b-a4c6-a74e77554e30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:47.816891 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-56423749-a179-446b-a4c6-a74e77554e30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:47.823917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56423749-a179-446b-a4c6-a74e77554e30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:47.824443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56423749-a179-446b-a4c6-a74e77554e30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:47.824878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56423749-a179-446b-a4c6-a74e77554e30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:47.825565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56423749-a179-446b-a4c6-a74e77554e30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:47.826016 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56423749-a179-446b-a4c6-a74e77554e30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:47.826524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-56423749-a179-446b-a4c6-a74e77554e30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:47.891256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-56423749-a179-446b-a4c6-a74e77554e30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:47.899671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-56423749-a179-446b-a4c6-a74e77554e30 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.087373 Sep 08 14:05:47.900233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1161/2207] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:47 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:48.123999 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a6b19a55-f0a7-426f-b568-dec63481b8e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:48.126516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a6b19a55-f0a7-426f-b568-dec63481b8e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:48.136177 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a6b19a55-f0a7-426f-b568-dec63481b8e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:48.136723 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a6b19a55-f0a7-426f-b568-dec63481b8e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:48.137201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a6b19a55-f0a7-426f-b568-dec63481b8e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:48.138493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a6b19a55-f0a7-426f-b568-dec63481b8e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:48.139118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a6b19a55-f0a7-426f-b568-dec63481b8e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:48.139827 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a6b19a55-f0a7-426f-b568-dec63481b8e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:48.173884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a6b19a55-f0a7-426f-b568-dec63481b8e9 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054064 Sep 08 14:05:48.174691 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1162/2208] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:48 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:48.189000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0fe6e309-d620-4a32-a8b1-5873565154d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:48.191642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0fe6e309-d620-4a32-a8b1-5873565154d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:48.197916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0fe6e309-d620-4a32-a8b1-5873565154d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:48.198451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0fe6e309-d620-4a32-a8b1-5873565154d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:48.198904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0fe6e309-d620-4a32-a8b1-5873565154d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:48.199611 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0fe6e309-d620-4a32-a8b1-5873565154d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:48.200072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0fe6e309-d620-4a32-a8b1-5873565154d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:48.200563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0fe6e309-d620-4a32-a8b1-5873565154d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:48.239389 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0fe6e309-d620-4a32-a8b1-5873565154d1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054331 Sep 08 14:05:48.242295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1163/2209] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:48 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:48.506903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0317aba7-f314-4fd2-85be-88fdb234c163 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:48.507420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0317aba7-f314-4fd2-85be-88fdb234c163 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:48.513509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0317aba7-f314-4fd2-85be-88fdb234c163 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:48.514028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0317aba7-f314-4fd2-85be-88fdb234c163 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:48.514503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0317aba7-f314-4fd2-85be-88fdb234c163 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:48.515196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0317aba7-f314-4fd2-85be-88fdb234c163 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:48.515646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0317aba7-f314-4fd2-85be-88fdb234c163 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:48.516072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0317aba7-f314-4fd2-85be-88fdb234c163 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:48.657309 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0317aba7-f314-4fd2-85be-88fdb234c163 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:48.665454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0317aba7-f314-4fd2-85be-88fdb234c163 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.162662 Sep 08 14:05:48.666063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1164/2210] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:48 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:48.924139 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4f8c65a8-e219-4c61-87d7-8ee71c4bfddc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:48.924797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4f8c65a8-e219-4c61-87d7-8ee71c4bfddc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:48.931871 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f8c65a8-e219-4c61-87d7-8ee71c4bfddc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:48.932359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f8c65a8-e219-4c61-87d7-8ee71c4bfddc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:48.932832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f8c65a8-e219-4c61-87d7-8ee71c4bfddc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:48.933580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f8c65a8-e219-4c61-87d7-8ee71c4bfddc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:48.934030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f8c65a8-e219-4c61-87d7-8ee71c4bfddc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:48.934487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f8c65a8-e219-4c61-87d7-8ee71c4bfddc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:49.021315 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4f8c65a8-e219-4c61-87d7-8ee71c4bfddc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:49.029691 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4f8c65a8-e219-4c61-87d7-8ee71c4bfddc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.109695 Sep 08 14:05:49.030268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1165/2211] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:48 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:49.187643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bebe4ef4-998d-4151-94cf-a66b9922ccc1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:49.190444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bebe4ef4-998d-4151-94cf-a66b9922ccc1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:49.199819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bebe4ef4-998d-4151-94cf-a66b9922ccc1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:49.200310 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bebe4ef4-998d-4151-94cf-a66b9922ccc1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:49.200840 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bebe4ef4-998d-4151-94cf-a66b9922ccc1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:49.201581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bebe4ef4-998d-4151-94cf-a66b9922ccc1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:49.202036 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bebe4ef4-998d-4151-94cf-a66b9922ccc1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:49.202512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bebe4ef4-998d-4151-94cf-a66b9922ccc1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:49.235130 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bebe4ef4-998d-4151-94cf-a66b9922ccc1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051682 Sep 08 14:05:49.235702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1166/2212] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:49 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:49.252833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d6e604a8-44a2-46c0-9eaf-ec6ef45ca242 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:49.255678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d6e604a8-44a2-46c0-9eaf-ec6ef45ca242 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:49.262692 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d6e604a8-44a2-46c0-9eaf-ec6ef45ca242 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:49.263200 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d6e604a8-44a2-46c0-9eaf-ec6ef45ca242 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:49.263641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d6e604a8-44a2-46c0-9eaf-ec6ef45ca242 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:49.264437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d6e604a8-44a2-46c0-9eaf-ec6ef45ca242 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:49.264894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d6e604a8-44a2-46c0-9eaf-ec6ef45ca242 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:49.265321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d6e604a8-44a2-46c0-9eaf-ec6ef45ca242 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:49.296580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d6e604a8-44a2-46c0-9eaf-ec6ef45ca242 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047744 Sep 08 14:05:49.297151 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1167/2213] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:49 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:49.701183 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-765fcab5-7cd8-4d50-8f27-334d25e3c0dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:49.701787 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-765fcab5-7cd8-4d50-8f27-334d25e3c0dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:49.714191 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-765fcab5-7cd8-4d50-8f27-334d25e3c0dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:49.714488 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-765fcab5-7cd8-4d50-8f27-334d25e3c0dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:49.715528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-765fcab5-7cd8-4d50-8f27-334d25e3c0dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:49.715864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-765fcab5-7cd8-4d50-8f27-334d25e3c0dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:49.716615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-765fcab5-7cd8-4d50-8f27-334d25e3c0dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:49.717358 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-765fcab5-7cd8-4d50-8f27-334d25e3c0dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:49.877221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-765fcab5-7cd8-4d50-8f27-334d25e3c0dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:49.886323 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-765fcab5-7cd8-4d50-8f27-334d25e3c0dd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.189106 Sep 08 14:05:49.886908 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1168/2214] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:49 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:50.059975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8a0bc26b-3557-4fa4-a35b-d2cd479c464a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:50.060766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8a0bc26b-3557-4fa4-a35b-d2cd479c464a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:50.067338 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a0bc26b-3557-4fa4-a35b-d2cd479c464a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:50.067830 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a0bc26b-3557-4fa4-a35b-d2cd479c464a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:50.068268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a0bc26b-3557-4fa4-a35b-d2cd479c464a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:50.068973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a0bc26b-3557-4fa4-a35b-d2cd479c464a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:50.069967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a0bc26b-3557-4fa4-a35b-d2cd479c464a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:50.070418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a0bc26b-3557-4fa4-a35b-d2cd479c464a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:50.150718 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8a0bc26b-3557-4fa4-a35b-d2cd479c464a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:50.160710 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8a0bc26b-3557-4fa4-a35b-d2cd479c464a tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.105201 Sep 08 14:05:50.161247 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1169/2215] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:50 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:50.248529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d23a501b-c83b-4ab7-bf1e-1a10a0ce95de tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:50.251420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d23a501b-c83b-4ab7-bf1e-1a10a0ce95de tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:50.261811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d23a501b-c83b-4ab7-bf1e-1a10a0ce95de tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:50.263049 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d23a501b-c83b-4ab7-bf1e-1a10a0ce95de tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:50.263606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d23a501b-c83b-4ab7-bf1e-1a10a0ce95de tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:50.264503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d23a501b-c83b-4ab7-bf1e-1a10a0ce95de tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:50.265068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d23a501b-c83b-4ab7-bf1e-1a10a0ce95de tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:50.265627 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d23a501b-c83b-4ab7-bf1e-1a10a0ce95de tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:50.300608 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d23a501b-c83b-4ab7-bf1e-1a10a0ce95de tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056324 Sep 08 14:05:50.301195 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1170/2216] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:50 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:50.310292 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4a5bd98f-e36f-4a44-87c0-1ea4c76020b4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:50.312563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4a5bd98f-e36f-4a44-87c0-1ea4c76020b4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:50.319024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4a5bd98f-e36f-4a44-87c0-1ea4c76020b4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:50.320048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4a5bd98f-e36f-4a44-87c0-1ea4c76020b4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:50.320481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4a5bd98f-e36f-4a44-87c0-1ea4c76020b4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:50.321183 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4a5bd98f-e36f-4a44-87c0-1ea4c76020b4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:50.321635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4a5bd98f-e36f-4a44-87c0-1ea4c76020b4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:50.322057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4a5bd98f-e36f-4a44-87c0-1ea4c76020b4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:50.354526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4a5bd98f-e36f-4a44-87c0-1ea4c76020b4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048340 Sep 08 14:05:50.355105 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1171/2217] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:50 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:50.921547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e33d801f-51f1-44a2-bb55-93b34cb75d49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:50.922447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e33d801f-51f1-44a2-bb55-93b34cb75d49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:50.932378 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e33d801f-51f1-44a2-bb55-93b34cb75d49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:50.933172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e33d801f-51f1-44a2-bb55-93b34cb75d49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:50.933888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e33d801f-51f1-44a2-bb55-93b34cb75d49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:50.935049 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e33d801f-51f1-44a2-bb55-93b34cb75d49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:50.935668 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e33d801f-51f1-44a2-bb55-93b34cb75d49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:50.936162 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e33d801f-51f1-44a2-bb55-93b34cb75d49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:51.100527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e33d801f-51f1-44a2-bb55-93b34cb75d49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:51.112155 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e33d801f-51f1-44a2-bb55-93b34cb75d49 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.196233 Sep 08 14:05:51.112742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1172/2218] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:50 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:51.190894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8999447c-7088-4066-b55d-27e044785690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:51.191378 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8999447c-7088-4066-b55d-27e044785690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:51.198061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8999447c-7088-4066-b55d-27e044785690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:51.198573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8999447c-7088-4066-b55d-27e044785690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:51.199030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8999447c-7088-4066-b55d-27e044785690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:51.199727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8999447c-7088-4066-b55d-27e044785690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:51.200175 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8999447c-7088-4066-b55d-27e044785690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:51.200600 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8999447c-7088-4066-b55d-27e044785690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:51.274811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8999447c-7088-4066-b55d-27e044785690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:51.283908 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8999447c-7088-4066-b55d-27e044785690 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.097588 Sep 08 14:05:51.284461 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1173/2219] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:51 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:51.313764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b42353a8-75f5-43d4-ba4e-5e1383c7a178 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:51.316272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b42353a8-75f5-43d4-ba4e-5e1383c7a178 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:51.324452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b42353a8-75f5-43d4-ba4e-5e1383c7a178 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:51.324942 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b42353a8-75f5-43d4-ba4e-5e1383c7a178 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:51.325380 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b42353a8-75f5-43d4-ba4e-5e1383c7a178 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:51.326094 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b42353a8-75f5-43d4-ba4e-5e1383c7a178 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:51.326594 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b42353a8-75f5-43d4-ba4e-5e1383c7a178 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:51.327017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b42353a8-75f5-43d4-ba4e-5e1383c7a178 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:51.363288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b42353a8-75f5-43d4-ba4e-5e1383c7a178 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053932 Sep 08 14:05:51.364468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1174/2220] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:51 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:05:51.371274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f1fe9b2c-2bea-49a8-980b-6171105797ec tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:51.373726 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f1fe9b2c-2bea-49a8-980b-6171105797ec tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:51.380144 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f1fe9b2c-2bea-49a8-980b-6171105797ec tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:51.380632 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f1fe9b2c-2bea-49a8-980b-6171105797ec tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:51.381084 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f1fe9b2c-2bea-49a8-980b-6171105797ec tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:51.382611 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f1fe9b2c-2bea-49a8-980b-6171105797ec tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:51.382946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f1fe9b2c-2bea-49a8-980b-6171105797ec tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:51.383376 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f1fe9b2c-2bea-49a8-980b-6171105797ec tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:51.417101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f1fe9b2c-2bea-49a8-980b-6171105797ec tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051429 Sep 08 14:05:51.417677 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1175/2221] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:51 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:52.139577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-05ff0dd9-a876-49c2-82f0-955aac2183f3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:52.140206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-05ff0dd9-a876-49c2-82f0-955aac2183f3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:52.147960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05ff0dd9-a876-49c2-82f0-955aac2183f3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:52.148451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05ff0dd9-a876-49c2-82f0-955aac2183f3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:52.148887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05ff0dd9-a876-49c2-82f0-955aac2183f3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:52.149586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05ff0dd9-a876-49c2-82f0-955aac2183f3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:52.150041 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05ff0dd9-a876-49c2-82f0-955aac2183f3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:52.150495 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-05ff0dd9-a876-49c2-82f0-955aac2183f3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:52.167639 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:52.315477 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-05ff0dd9-a876-49c2-82f0-955aac2183f3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:52.325499 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-05ff0dd9-a876-49c2-82f0-955aac2183f3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.190417 Sep 08 14:05:52.326090 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1176/2222] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:52 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:52.331132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f8e7f39e-269d-44ef-8d50-f3c9c9c5ed57 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:52.331729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f8e7f39e-269d-44ef-8d50-f3c9c9c5ed57 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:52.339920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8e7f39e-269d-44ef-8d50-f3c9c9c5ed57 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:52.340396 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8e7f39e-269d-44ef-8d50-f3c9c9c5ed57 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:52.340840 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8e7f39e-269d-44ef-8d50-f3c9c9c5ed57 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:52.341600 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8e7f39e-269d-44ef-8d50-f3c9c9c5ed57 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:52.342180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8e7f39e-269d-44ef-8d50-f3c9c9c5ed57 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:52.345559 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f8e7f39e-269d-44ef-8d50-f3c9c9c5ed57 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:52.412948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f8e7f39e-269d-44ef-8d50-f3c9c9c5ed57 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:52.423523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f8e7f39e-269d-44ef-8d50-f3c9c9c5ed57 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.096126 Sep 08 14:05:52.424337 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1177/2223] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:52 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:52.431628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9f0a54ae-b5bd-4c2a-8b72-9cbf4f671614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:52.434764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9f0a54ae-b5bd-4c2a-8b72-9cbf4f671614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:52.443781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9f0a54ae-b5bd-4c2a-8b72-9cbf4f671614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:52.444908 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9f0a54ae-b5bd-4c2a-8b72-9cbf4f671614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:52.445407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9f0a54ae-b5bd-4c2a-8b72-9cbf4f671614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:52.446130 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9f0a54ae-b5bd-4c2a-8b72-9cbf4f671614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:52.446616 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9f0a54ae-b5bd-4c2a-8b72-9cbf4f671614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:52.447042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9f0a54ae-b5bd-4c2a-8b72-9cbf4f671614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:52.485031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9f0a54ae-b5bd-4c2a-8b72-9cbf4f671614 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059288 Sep 08 14:05:52.485621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1178/2224] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:52 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:52.490877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e3178d86-2e5d-43a5-b796-cf6be3975e81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:52.493091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e3178d86-2e5d-43a5-b796-cf6be3975e81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:52.502760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e3178d86-2e5d-43a5-b796-cf6be3975e81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:52.503275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e3178d86-2e5d-43a5-b796-cf6be3975e81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:52.503574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e3178d86-2e5d-43a5-b796-cf6be3975e81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:52.504287 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e3178d86-2e5d-43a5-b796-cf6be3975e81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:52.504746 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e3178d86-2e5d-43a5-b796-cf6be3975e81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:52.505175 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e3178d86-2e5d-43a5-b796-cf6be3975e81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:52.538876 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e3178d86-2e5d-43a5-b796-cf6be3975e81 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052121 Sep 08 14:05:52.539503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1179/2225] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:52 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:53.356171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8dfc7d16-a606-404e-8a31-7bad29df4727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:53.357068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8dfc7d16-a606-404e-8a31-7bad29df4727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:53.363718 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8dfc7d16-a606-404e-8a31-7bad29df4727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:53.364398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8dfc7d16-a606-404e-8a31-7bad29df4727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:53.364876 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8dfc7d16-a606-404e-8a31-7bad29df4727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:53.366493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8dfc7d16-a606-404e-8a31-7bad29df4727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:53.367042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8dfc7d16-a606-404e-8a31-7bad29df4727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:53.367467 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8dfc7d16-a606-404e-8a31-7bad29df4727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:53.520274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8dfc7d16-a606-404e-8a31-7bad29df4727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:53.529812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8dfc7d16-a606-404e-8a31-7bad29df4727 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.178304 Sep 08 14:05:53.530499 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1180/2226] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:53 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:53.535270 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c552b1de-bea0-4f8f-ad5c-a47d216797ce tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:53.535875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c552b1de-bea0-4f8f-ad5c-a47d216797ce tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:53.542838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c552b1de-bea0-4f8f-ad5c-a47d216797ce tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:53.543328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c552b1de-bea0-4f8f-ad5c-a47d216797ce tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:53.543832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c552b1de-bea0-4f8f-ad5c-a47d216797ce tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:53.544531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c552b1de-bea0-4f8f-ad5c-a47d216797ce tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:53.544979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c552b1de-bea0-4f8f-ad5c-a47d216797ce tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:53.545402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c552b1de-bea0-4f8f-ad5c-a47d216797ce tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:53.610978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c552b1de-bea0-4f8f-ad5c-a47d216797ce tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:53.619316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c552b1de-bea0-4f8f-ad5c-a47d216797ce tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.087822 Sep 08 14:05:53.619887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1181/2227] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:53 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:53.624559 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cd2c8634-89d4-4512-81ab-d5327a177d3c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:53.627070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cd2c8634-89d4-4512-81ab-d5327a177d3c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:53.633299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cd2c8634-89d4-4512-81ab-d5327a177d3c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:53.633829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cd2c8634-89d4-4512-81ab-d5327a177d3c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:53.634456 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cd2c8634-89d4-4512-81ab-d5327a177d3c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:53.635200 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cd2c8634-89d4-4512-81ab-d5327a177d3c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:53.635654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cd2c8634-89d4-4512-81ab-d5327a177d3c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:53.636072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cd2c8634-89d4-4512-81ab-d5327a177d3c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:53.670638 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cd2c8634-89d4-4512-81ab-d5327a177d3c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049541 Sep 08 14:05:53.671230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1182/2228] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:53 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:53.675942 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-58eb7e10-345b-4c2b-bb3d-57338e540735 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:53.678158 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-58eb7e10-345b-4c2b-bb3d-57338e540735 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:53.684640 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58eb7e10-345b-4c2b-bb3d-57338e540735 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:53.685197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58eb7e10-345b-4c2b-bb3d-57338e540735 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:53.685642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58eb7e10-345b-4c2b-bb3d-57338e540735 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:53.686428 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58eb7e10-345b-4c2b-bb3d-57338e540735 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:53.686887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58eb7e10-345b-4c2b-bb3d-57338e540735 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:53.687311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58eb7e10-345b-4c2b-bb3d-57338e540735 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:53.720235 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-58eb7e10-345b-4c2b-bb3d-57338e540735 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047904 Sep 08 14:05:53.720833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1183/2229] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:53 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:54.556204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b3330c4e-84df-436d-9540-999f9068eb01 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:54.556973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b3330c4e-84df-436d-9540-999f9068eb01 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:54.566704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b3330c4e-84df-436d-9540-999f9068eb01 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:54.567462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b3330c4e-84df-436d-9540-999f9068eb01 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:54.567914 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b3330c4e-84df-436d-9540-999f9068eb01 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:54.568623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b3330c4e-84df-436d-9540-999f9068eb01 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:54.569119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b3330c4e-84df-436d-9540-999f9068eb01 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:54.570167 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b3330c4e-84df-436d-9540-999f9068eb01 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:54.739442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b3330c4e-84df-436d-9540-999f9068eb01 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:54.748635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b3330c4e-84df-436d-9540-999f9068eb01 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.196707 Sep 08 14:05:54.749297 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1184/2230] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:54 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:54.758021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-695d1223-5dd3-4f10-98e5-05370f6b2f56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:54.759413 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-695d1223-5dd3-4f10-98e5-05370f6b2f56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:54.770460 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-695d1223-5dd3-4f10-98e5-05370f6b2f56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:54.771356 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-695d1223-5dd3-4f10-98e5-05370f6b2f56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:54.772150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-695d1223-5dd3-4f10-98e5-05370f6b2f56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:54.773447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-695d1223-5dd3-4f10-98e5-05370f6b2f56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:54.774303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-695d1223-5dd3-4f10-98e5-05370f6b2f56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:54.775060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-695d1223-5dd3-4f10-98e5-05370f6b2f56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:54.844416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-695d1223-5dd3-4f10-98e5-05370f6b2f56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:54.853502 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-695d1223-5dd3-4f10-98e5-05370f6b2f56 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.102546 Sep 08 14:05:54.854163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1185/2231] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:54 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:54.860454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-25d84bdb-dd03-4325-9612-fec25f0d1f7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:54.863758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-25d84bdb-dd03-4325-9612-fec25f0d1f7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:54.871879 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25d84bdb-dd03-4325-9612-fec25f0d1f7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:54.872361 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25d84bdb-dd03-4325-9612-fec25f0d1f7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:54.872795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25d84bdb-dd03-4325-9612-fec25f0d1f7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:54.873484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25d84bdb-dd03-4325-9612-fec25f0d1f7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:54.873936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25d84bdb-dd03-4325-9612-fec25f0d1f7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:54.874392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25d84bdb-dd03-4325-9612-fec25f0d1f7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:54.915171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-25d84bdb-dd03-4325-9612-fec25f0d1f7c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059929 Sep 08 14:05:54.915978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1186/2232] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:54 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:54.920523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4088c82c-88d6-41f9-9dfd-9b3e3c04630a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:54.923326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4088c82c-88d6-41f9-9dfd-9b3e3c04630a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:54.933481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4088c82c-88d6-41f9-9dfd-9b3e3c04630a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:54.933481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4088c82c-88d6-41f9-9dfd-9b3e3c04630a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:54.933481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4088c82c-88d6-41f9-9dfd-9b3e3c04630a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:54.933481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4088c82c-88d6-41f9-9dfd-9b3e3c04630a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:54.934095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4088c82c-88d6-41f9-9dfd-9b3e3c04630a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:54.934095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4088c82c-88d6-41f9-9dfd-9b3e3c04630a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:54.969810 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4088c82c-88d6-41f9-9dfd-9b3e3c04630a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052982 Sep 08 14:05:54.970448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1187/2233] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:54 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:55.138816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=63 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:55.175871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=52 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:55.791636 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f24c072b-6c68-4b18-9c64-d7c7354064ea tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:55.792285 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f24c072b-6c68-4b18-9c64-d7c7354064ea tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:55.799170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f24c072b-6c68-4b18-9c64-d7c7354064ea tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:55.799658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f24c072b-6c68-4b18-9c64-d7c7354064ea tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:55.800117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f24c072b-6c68-4b18-9c64-d7c7354064ea tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:55.800822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f24c072b-6c68-4b18-9c64-d7c7354064ea tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:55.801274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f24c072b-6c68-4b18-9c64-d7c7354064ea tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:55.801698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f24c072b-6c68-4b18-9c64-d7c7354064ea tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:55.985667 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f24c072b-6c68-4b18-9c64-d7c7354064ea tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:55.995024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f24c072b-6c68-4b18-9c64-d7c7354064ea tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.209121 Sep 08 14:05:55.995611 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1188/2234] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:55 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:56.000496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-14818907-0173-4858-a259-ce129a26206c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:56.001148 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-14818907-0173-4858-a259-ce129a26206c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:56.010880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-14818907-0173-4858-a259-ce129a26206c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:56.011451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-14818907-0173-4858-a259-ce129a26206c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:56.012042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-14818907-0173-4858-a259-ce129a26206c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:56.013138 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-14818907-0173-4858-a259-ce129a26206c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:56.013641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-14818907-0173-4858-a259-ce129a26206c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:56.014118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-14818907-0173-4858-a259-ce129a26206c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:56.093365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-14818907-0173-4858-a259-ce129a26206c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:56.103909 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-14818907-0173-4858-a259-ce129a26206c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.106792 Sep 08 14:05:56.104960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1189/2235] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:55 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:56.110280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bd1dc37e-7db9-43a7-b623-4edae89171e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:56.112472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bd1dc37e-7db9-43a7-b623-4edae89171e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:56.114591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:56.120110 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd1dc37e-7db9-43a7-b623-4edae89171e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:56.120689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd1dc37e-7db9-43a7-b623-4edae89171e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:56.121290 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd1dc37e-7db9-43a7-b623-4edae89171e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:56.121957 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd1dc37e-7db9-43a7-b623-4edae89171e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:56.122450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd1dc37e-7db9-43a7-b623-4edae89171e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:56.122882 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd1dc37e-7db9-43a7-b623-4edae89171e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:56.162269 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bd1dc37e-7db9-43a7-b623-4edae89171e4 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056010 Sep 08 14:05:56.163124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1190/2236] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:56 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:56.168491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ae1f1381-6f7c-41fb-8953-d656e1764387 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:56.170780 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ae1f1381-6f7c-41fb-8953-d656e1764387 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:56.177590 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae1f1381-6f7c-41fb-8953-d656e1764387 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:56.178161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae1f1381-6f7c-41fb-8953-d656e1764387 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:56.178617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae1f1381-6f7c-41fb-8953-d656e1764387 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:56.179323 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae1f1381-6f7c-41fb-8953-d656e1764387 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:56.179777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae1f1381-6f7c-41fb-8953-d656e1764387 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:56.180221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae1f1381-6f7c-41fb-8953-d656e1764387 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:56.218209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ae1f1381-6f7c-41fb-8953-d656e1764387 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053395 Sep 08 14:05:56.218610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1191/2237] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:56 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:57.033462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-36675bb0-ce81-424f-ad40-baa39c363585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:57.034098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-36675bb0-ce81-424f-ad40-baa39c363585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:57.041931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-36675bb0-ce81-424f-ad40-baa39c363585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:57.042520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-36675bb0-ce81-424f-ad40-baa39c363585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:57.043014 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-36675bb0-ce81-424f-ad40-baa39c363585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:57.043900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-36675bb0-ce81-424f-ad40-baa39c363585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:57.044591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-36675bb0-ce81-424f-ad40-baa39c363585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:57.044952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-36675bb0-ce81-424f-ad40-baa39c363585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:57.195724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-36675bb0-ce81-424f-ad40-baa39c363585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:57.204451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-36675bb0-ce81-424f-ad40-baa39c363585 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.175276 Sep 08 14:05:57.205022 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1192/2238] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:57 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:57.209508 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-eceefbf4-2e5d-4422-a4d4-2b86b4ab994c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:57.210670 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-eceefbf4-2e5d-4422-a4d4-2b86b4ab994c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:57.218424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eceefbf4-2e5d-4422-a4d4-2b86b4ab994c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:57.218919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eceefbf4-2e5d-4422-a4d4-2b86b4ab994c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:57.219373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eceefbf4-2e5d-4422-a4d4-2b86b4ab994c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:57.220072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eceefbf4-2e5d-4422-a4d4-2b86b4ab994c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:57.220535 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eceefbf4-2e5d-4422-a4d4-2b86b4ab994c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:57.220991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eceefbf4-2e5d-4422-a4d4-2b86b4ab994c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:57.223830 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:57.289428 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-eceefbf4-2e5d-4422-a4d4-2b86b4ab994c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:57.305803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-eceefbf4-2e5d-4422-a4d4-2b86b4ab994c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.097279 Sep 08 14:05:57.306745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1193/2239] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:57 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:57.311965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f684f5e4-29c6-4621-b372-3d8ce7384652 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:57.314101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f684f5e4-29c6-4621-b372-3d8ce7384652 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:57.322106 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f684f5e4-29c6-4621-b372-3d8ce7384652 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:57.322685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f684f5e4-29c6-4621-b372-3d8ce7384652 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:57.323125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f684f5e4-29c6-4621-b372-3d8ce7384652 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:57.323853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f684f5e4-29c6-4621-b372-3d8ce7384652 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:57.324314 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f684f5e4-29c6-4621-b372-3d8ce7384652 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:57.324886 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f684f5e4-29c6-4621-b372-3d8ce7384652 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:57.363376 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f684f5e4-29c6-4621-b372-3d8ce7384652 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055019 Sep 08 14:05:57.364400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1194/2240] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:57 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:57.370783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-eeb8a2df-fa09-449e-9d44-6db1d7c8d589 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:57.372936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-eeb8a2df-fa09-449e-9d44-6db1d7c8d589 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:57.381350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eeb8a2df-fa09-449e-9d44-6db1d7c8d589 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:57.381984 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eeb8a2df-fa09-449e-9d44-6db1d7c8d589 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:57.382470 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eeb8a2df-fa09-449e-9d44-6db1d7c8d589 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:57.383176 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eeb8a2df-fa09-449e-9d44-6db1d7c8d589 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:57.383723 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eeb8a2df-fa09-449e-9d44-6db1d7c8d589 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:57.384232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eeb8a2df-fa09-449e-9d44-6db1d7c8d589 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:57.421237 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-eeb8a2df-fa09-449e-9d44-6db1d7c8d589 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055476 Sep 08 14:05:57.421808 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1195/2241] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:57 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:58.232350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6d83cf16-6270-470b-84ce-c781e33e6167 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:58.233376 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6d83cf16-6270-470b-84ce-c781e33e6167 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:58.240042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6d83cf16-6270-470b-84ce-c781e33e6167 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:58.240797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6d83cf16-6270-470b-84ce-c781e33e6167 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:58.241763 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6d83cf16-6270-470b-84ce-c781e33e6167 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:58.242510 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6d83cf16-6270-470b-84ce-c781e33e6167 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:58.243025 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6d83cf16-6270-470b-84ce-c781e33e6167 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:58.243397 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6d83cf16-6270-470b-84ce-c781e33e6167 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:58.423774 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6d83cf16-6270-470b-84ce-c781e33e6167 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:58.432323 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6d83cf16-6270-470b-84ce-c781e33e6167 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.203820 Sep 08 14:05:58.432905 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1196/2242] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:58 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:58.437540 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ca347b84-7772-4990-911d-042ee151fbcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:58.438170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ca347b84-7772-4990-911d-042ee151fbcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:58.447657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca347b84-7772-4990-911d-042ee151fbcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:58.448252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca347b84-7772-4990-911d-042ee151fbcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:58.448754 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca347b84-7772-4990-911d-042ee151fbcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:58.449725 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca347b84-7772-4990-911d-042ee151fbcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:58.450535 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca347b84-7772-4990-911d-042ee151fbcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:58.451231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca347b84-7772-4990-911d-042ee151fbcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:58.520823 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ca347b84-7772-4990-911d-042ee151fbcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:58.529384 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ca347b84-7772-4990-911d-042ee151fbcc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.095515 Sep 08 14:05:58.529951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1197/2243] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:58 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:58.535321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ff6f35b0-4aa0-49d0-afd2-6831fd29e9ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:58.537989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ff6f35b0-4aa0-49d0-afd2-6831fd29e9ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:58.544476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ff6f35b0-4aa0-49d0-afd2-6831fd29e9ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:58.544963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ff6f35b0-4aa0-49d0-afd2-6831fd29e9ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:58.545400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ff6f35b0-4aa0-49d0-afd2-6831fd29e9ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:58.546095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ff6f35b0-4aa0-49d0-afd2-6831fd29e9ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:58.546610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ff6f35b0-4aa0-49d0-afd2-6831fd29e9ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:58.547004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ff6f35b0-4aa0-49d0-afd2-6831fd29e9ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:58.579831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ff6f35b0-4aa0-49d0-afd2-6831fd29e9ea tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048609 Sep 08 14:05:58.580427 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1198/2244] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:58 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:58.589308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-dcceed67-1b3a-4237-8033-8f1227f1cffe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:58.591784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-dcceed67-1b3a-4237-8033-8f1227f1cffe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:58.598112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dcceed67-1b3a-4237-8033-8f1227f1cffe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:58.598639 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dcceed67-1b3a-4237-8033-8f1227f1cffe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:58.599077 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dcceed67-1b3a-4237-8033-8f1227f1cffe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:58.599774 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dcceed67-1b3a-4237-8033-8f1227f1cffe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:58.600282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dcceed67-1b3a-4237-8033-8f1227f1cffe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:58.600704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dcceed67-1b3a-4237-8033-8f1227f1cffe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:58.643845 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-dcceed67-1b3a-4237-8033-8f1227f1cffe tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061843 Sep 08 14:05:58.644428 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1199/2245] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:58 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:59.459364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-33545af6-c569-46e0-95a3-62276b807aca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:59.460850 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-33545af6-c569-46e0-95a3-62276b807aca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:59.467836 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-33545af6-c569-46e0-95a3-62276b807aca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:59.468513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-33545af6-c569-46e0-95a3-62276b807aca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:59.468952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-33545af6-c569-46e0-95a3-62276b807aca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:59.469824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-33545af6-c569-46e0-95a3-62276b807aca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:59.470305 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-33545af6-c569-46e0-95a3-62276b807aca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:59.470730 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-33545af6-c569-46e0-95a3-62276b807aca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:59.631712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-33545af6-c569-46e0-95a3-62276b807aca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:59.640129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-33545af6-c569-46e0-95a3-62276b807aca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.185435 Sep 08 14:05:59.641140 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1200/2246] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:59 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:05:59.646300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f0b7cb58-3db3-4faa-b644-92ce1a76752c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:59.646910 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f0b7cb58-3db3-4faa-b644-92ce1a76752c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:59.658386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0b7cb58-3db3-4faa-b644-92ce1a76752c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:59.658881 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0b7cb58-3db3-4faa-b644-92ce1a76752c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:59.659317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0b7cb58-3db3-4faa-b644-92ce1a76752c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:59.660408 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0b7cb58-3db3-4faa-b644-92ce1a76752c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:59.660881 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0b7cb58-3db3-4faa-b644-92ce1a76752c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:59.661306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0b7cb58-3db3-4faa-b644-92ce1a76752c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:59.742656 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f0b7cb58-3db3-4faa-b644-92ce1a76752c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:05:59.754668 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f0b7cb58-3db3-4faa-b644-92ce1a76752c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.112124 Sep 08 14:05:59.755356 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1201/2247] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:05:59 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:05:59.760411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-df1fa466-f33e-4a8b-b321-111c9e4e1e36 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:59.762584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-df1fa466-f33e-4a8b-b321-111c9e4e1e36 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:59.771328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df1fa466-f33e-4a8b-b321-111c9e4e1e36 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:59.771913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df1fa466-f33e-4a8b-b321-111c9e4e1e36 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:59.772889 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df1fa466-f33e-4a8b-b321-111c9e4e1e36 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:59.773602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df1fa466-f33e-4a8b-b321-111c9e4e1e36 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:59.774051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df1fa466-f33e-4a8b-b321-111c9e4e1e36 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:59.774637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df1fa466-f33e-4a8b-b321-111c9e4e1e36 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:59.808435 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-df1fa466-f33e-4a8b-b321-111c9e4e1e36 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051992 Sep 08 14:05:59.809011 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1202/2248] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:59 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:05:59.813589 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-eda2ca9f-6217-49ec-88c4-f8984df28ed4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:05:59.816058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-eda2ca9f-6217-49ec-88c4-f8984df28ed4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:05:59.823070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eda2ca9f-6217-49ec-88c4-f8984df28ed4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:59.823565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eda2ca9f-6217-49ec-88c4-f8984df28ed4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:59.824012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eda2ca9f-6217-49ec-88c4-f8984df28ed4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:59.824712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eda2ca9f-6217-49ec-88c4-f8984df28ed4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:05:59.825164 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eda2ca9f-6217-49ec-88c4-f8984df28ed4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:05:59.825679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eda2ca9f-6217-49ec-88c4-f8984df28ed4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:05:59.857700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-eda2ca9f-6217-49ec-88c4-f8984df28ed4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047626 Sep 08 14:05:59.858493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1203/2249] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:05:59 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:00.666425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c3c9f5c2-112a-44c4-8df3-17b33e48f1bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:00.667159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c3c9f5c2-112a-44c4-8df3-17b33e48f1bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:00.673905 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3c9f5c2-112a-44c4-8df3-17b33e48f1bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:00.674574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3c9f5c2-112a-44c4-8df3-17b33e48f1bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:00.674998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3c9f5c2-112a-44c4-8df3-17b33e48f1bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:00.675716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3c9f5c2-112a-44c4-8df3-17b33e48f1bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:00.676164 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3c9f5c2-112a-44c4-8df3-17b33e48f1bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:00.676621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c3c9f5c2-112a-44c4-8df3-17b33e48f1bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:00.825184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c3c9f5c2-112a-44c4-8df3-17b33e48f1bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:00.834013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c3c9f5c2-112a-44c4-8df3-17b33e48f1bc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.171886 Sep 08 14:06:00.834748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1204/2250] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:00 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:00.843179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-11a9978e-edfe-49e0-b3b3-a9905527ef82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:00.843931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-11a9978e-edfe-49e0-b3b3-a9905527ef82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:00.853970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-11a9978e-edfe-49e0-b3b3-a9905527ef82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:00.854786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-11a9978e-edfe-49e0-b3b3-a9905527ef82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:00.855490 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-11a9978e-edfe-49e0-b3b3-a9905527ef82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:00.856609 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-11a9978e-edfe-49e0-b3b3-a9905527ef82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:00.857367 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-11a9978e-edfe-49e0-b3b3-a9905527ef82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:00.858051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-11a9978e-edfe-49e0-b3b3-a9905527ef82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:00.931302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-11a9978e-edfe-49e0-b3b3-a9905527ef82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:00.945930 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-11a9978e-edfe-49e0-b3b3-a9905527ef82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.108837 Sep 08 14:06:00.946998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1205/2251] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:00 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:00.955566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-39649557-514e-4623-959b-8ae5bba13b5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:00.960150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-39649557-514e-4623-959b-8ae5bba13b5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:00.970331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-39649557-514e-4623-959b-8ae5bba13b5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:00.971210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-39649557-514e-4623-959b-8ae5bba13b5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:00.971954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-39649557-514e-4623-959b-8ae5bba13b5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:00.973159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-39649557-514e-4623-959b-8ae5bba13b5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:00.973986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-39649557-514e-4623-959b-8ae5bba13b5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:00.974826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-39649557-514e-4623-959b-8ae5bba13b5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:01.032581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-39649557-514e-4623-959b-8ae5bba13b5c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.083704 Sep 08 14:06:01.033392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1206/2252] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:00 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:01.039601 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b90cf7b2-98b4-475b-ac43-982667be9aa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:01.041783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b90cf7b2-98b4-475b-ac43-982667be9aa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:01.047964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b90cf7b2-98b4-475b-ac43-982667be9aa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:01.048465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b90cf7b2-98b4-475b-ac43-982667be9aa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:01.048907 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b90cf7b2-98b4-475b-ac43-982667be9aa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:01.049778 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b90cf7b2-98b4-475b-ac43-982667be9aa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:01.050210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b90cf7b2-98b4-475b-ac43-982667be9aa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:01.050649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b90cf7b2-98b4-475b-ac43-982667be9aa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:01.102043 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b90cf7b2-98b4-475b-ac43-982667be9aa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067384 Sep 08 14:06:01.102786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1207/2253] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:01 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:01.860247 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c928de89-1901-46a0-90f8-477cad170e55 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:01.861308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c928de89-1901-46a0-90f8-477cad170e55 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:01.870963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c928de89-1901-46a0-90f8-477cad170e55 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:01.871676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c928de89-1901-46a0-90f8-477cad170e55 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:01.872310 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c928de89-1901-46a0-90f8-477cad170e55 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:01.873181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c928de89-1901-46a0-90f8-477cad170e55 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:01.873727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c928de89-1901-46a0-90f8-477cad170e55 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:01.874354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c928de89-1901-46a0-90f8-477cad170e55 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:02.037119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c928de89-1901-46a0-90f8-477cad170e55 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:02.043205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c928de89-1901-46a0-90f8-477cad170e55 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.186940 Sep 08 14:06:02.043794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1208/2254] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:01 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:02.050274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-25dd3e0f-08e0-4c46-a971-827b5dd87f6f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:02.050949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-25dd3e0f-08e0-4c46-a971-827b5dd87f6f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:02.058080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25dd3e0f-08e0-4c46-a971-827b5dd87f6f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:02.058623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25dd3e0f-08e0-4c46-a971-827b5dd87f6f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:02.059069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25dd3e0f-08e0-4c46-a971-827b5dd87f6f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:02.059773 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25dd3e0f-08e0-4c46-a971-827b5dd87f6f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:02.060229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25dd3e0f-08e0-4c46-a971-827b5dd87f6f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:02.060654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-25dd3e0f-08e0-4c46-a971-827b5dd87f6f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:02.126577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-25dd3e0f-08e0-4c46-a971-827b5dd87f6f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:02.128746 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:02.135664 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-25dd3e0f-08e0-4c46-a971-827b5dd87f6f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.090787 Sep 08 14:06:02.136637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1209/2255] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:02 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:02.141163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4352e833-06a6-455c-a05e-851dfad0a99e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:02.146333 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4352e833-06a6-455c-a05e-851dfad0a99e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:02.151078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4352e833-06a6-455c-a05e-851dfad0a99e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:02.151938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4352e833-06a6-455c-a05e-851dfad0a99e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:02.152714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4352e833-06a6-455c-a05e-851dfad0a99e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:02.153779 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4352e833-06a6-455c-a05e-851dfad0a99e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:02.154685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4352e833-06a6-455c-a05e-851dfad0a99e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:02.155441 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4352e833-06a6-455c-a05e-851dfad0a99e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:02.192533 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4352e833-06a6-455c-a05e-851dfad0a99e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055132 Sep 08 14:06:02.193119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1210/2256] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:02 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:02.197981 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-77e6c5fc-d796-4661-8b81-3f3c1ccb9880 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:02.200181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-77e6c5fc-d796-4661-8b81-3f3c1ccb9880 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:02.206643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77e6c5fc-d796-4661-8b81-3f3c1ccb9880 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:02.207132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77e6c5fc-d796-4661-8b81-3f3c1ccb9880 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:02.207696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77e6c5fc-d796-4661-8b81-3f3c1ccb9880 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:02.208411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77e6c5fc-d796-4661-8b81-3f3c1ccb9880 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:02.208864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77e6c5fc-d796-4661-8b81-3f3c1ccb9880 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:02.209287 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77e6c5fc-d796-4661-8b81-3f3c1ccb9880 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:02.243939 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-77e6c5fc-d796-4661-8b81-3f3c1ccb9880 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049730 Sep 08 14:06:02.244533 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1211/2257] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:02 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:03.068218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a599ac54-4062-4e4d-a05b-b07587ca895f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:03.069415 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a599ac54-4062-4e4d-a05b-b07587ca895f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:03.076099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a599ac54-4062-4e4d-a05b-b07587ca895f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:03.076596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a599ac54-4062-4e4d-a05b-b07587ca895f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:03.077027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a599ac54-4062-4e4d-a05b-b07587ca895f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:03.077717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a599ac54-4062-4e4d-a05b-b07587ca895f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:03.078185 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a599ac54-4062-4e4d-a05b-b07587ca895f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:03.078685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a599ac54-4062-4e4d-a05b-b07587ca895f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:03.260550 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a599ac54-4062-4e4d-a05b-b07587ca895f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:03.274697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a599ac54-4062-4e4d-a05b-b07587ca895f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.210072 Sep 08 14:06:03.275598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1212/2258] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:03 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:03.289844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-436e47ed-6c2c-490c-9a96-15107fba37ee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:03.289844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-436e47ed-6c2c-490c-9a96-15107fba37ee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:03.291891 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-436e47ed-6c2c-490c-9a96-15107fba37ee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:03.292452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-436e47ed-6c2c-490c-9a96-15107fba37ee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:03.292899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-436e47ed-6c2c-490c-9a96-15107fba37ee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:03.293806 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-436e47ed-6c2c-490c-9a96-15107fba37ee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:03.294319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-436e47ed-6c2c-490c-9a96-15107fba37ee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:03.294822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-436e47ed-6c2c-490c-9a96-15107fba37ee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:03.397644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-436e47ed-6c2c-490c-9a96-15107fba37ee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:03.407416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-436e47ed-6c2c-490c-9a96-15107fba37ee tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.127542 Sep 08 14:06:03.408014 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1213/2259] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:03 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:03.415322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-adef3eb7-6ed8-4e70-9cc2-f679dcf3f01a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:03.418710 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-adef3eb7-6ed8-4e70-9cc2-f679dcf3f01a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:03.426425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-adef3eb7-6ed8-4e70-9cc2-f679dcf3f01a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:03.426960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-adef3eb7-6ed8-4e70-9cc2-f679dcf3f01a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:03.427437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-adef3eb7-6ed8-4e70-9cc2-f679dcf3f01a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:03.428192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-adef3eb7-6ed8-4e70-9cc2-f679dcf3f01a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:03.428668 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-adef3eb7-6ed8-4e70-9cc2-f679dcf3f01a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:03.429079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-adef3eb7-6ed8-4e70-9cc2-f679dcf3f01a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:03.461364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-adef3eb7-6ed8-4e70-9cc2-f679dcf3f01a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052019 Sep 08 14:06:03.461949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1214/2260] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:03 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:03.467345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-978211e1-8e04-4761-9b2d-5d9e9dfd022a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:03.469523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-978211e1-8e04-4761-9b2d-5d9e9dfd022a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:03.476494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-978211e1-8e04-4761-9b2d-5d9e9dfd022a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:03.476980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-978211e1-8e04-4761-9b2d-5d9e9dfd022a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:03.477419 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-978211e1-8e04-4761-9b2d-5d9e9dfd022a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:03.478121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-978211e1-8e04-4761-9b2d-5d9e9dfd022a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:03.478601 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-978211e1-8e04-4761-9b2d-5d9e9dfd022a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:03.479025 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-978211e1-8e04-4761-9b2d-5d9e9dfd022a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:03.510208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-978211e1-8e04-4761-9b2d-5d9e9dfd022a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046537 Sep 08 14:06:03.510797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1215/2261] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:03 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:04.303922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cd0b933c-1660-4513-8e70-e21dd9a37c31 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:04.304824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cd0b933c-1660-4513-8e70-e21dd9a37c31 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:04.311359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cd0b933c-1660-4513-8e70-e21dd9a37c31 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:04.311861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cd0b933c-1660-4513-8e70-e21dd9a37c31 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:04.312302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cd0b933c-1660-4513-8e70-e21dd9a37c31 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:04.313009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cd0b933c-1660-4513-8e70-e21dd9a37c31 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:04.313472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cd0b933c-1660-4513-8e70-e21dd9a37c31 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:04.314015 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cd0b933c-1660-4513-8e70-e21dd9a37c31 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:04.699309 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cd0b933c-1660-4513-8e70-e21dd9a37c31 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:04.710099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cd0b933c-1660-4513-8e70-e21dd9a37c31 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.410719 Sep 08 14:06:04.710728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1216/2262] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:04 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:04.715798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bcf72dd4-8075-4f0b-9558-e0f2cd583e36 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:04.716405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bcf72dd4-8075-4f0b-9558-e0f2cd583e36 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:04.723322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcf72dd4-8075-4f0b-9558-e0f2cd583e36 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:04.723819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcf72dd4-8075-4f0b-9558-e0f2cd583e36 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:04.724516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcf72dd4-8075-4f0b-9558-e0f2cd583e36 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:04.725754 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcf72dd4-8075-4f0b-9558-e0f2cd583e36 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:04.726234 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcf72dd4-8075-4f0b-9558-e0f2cd583e36 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:04.726661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcf72dd4-8075-4f0b-9558-e0f2cd583e36 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:04.795296 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bcf72dd4-8075-4f0b-9558-e0f2cd583e36 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:04.803655 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bcf72dd4-8075-4f0b-9558-e0f2cd583e36 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.091810 Sep 08 14:06:04.804246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1217/2263] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:04 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:04.809034 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c5012441-9947-4531-ae9d-f23953a4dd01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:04.811268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c5012441-9947-4531-ae9d-f23953a4dd01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:04.818162 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5012441-9947-4531-ae9d-f23953a4dd01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:04.818707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5012441-9947-4531-ae9d-f23953a4dd01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:04.824068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5012441-9947-4531-ae9d-f23953a4dd01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:04.824068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5012441-9947-4531-ae9d-f23953a4dd01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:04.824068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5012441-9947-4531-ae9d-f23953a4dd01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:04.824068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5012441-9947-4531-ae9d-f23953a4dd01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:04.860522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c5012441-9947-4531-ae9d-f23953a4dd01 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055192 Sep 08 14:06:04.861394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1218/2264] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:04 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:04.867807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-285eb383-e853-469f-8fda-51717a026f78 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:04.870471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-285eb383-e853-469f-8fda-51717a026f78 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:04.877395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-285eb383-e853-469f-8fda-51717a026f78 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:04.877778 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-285eb383-e853-469f-8fda-51717a026f78 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:04.878250 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-285eb383-e853-469f-8fda-51717a026f78 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:04.878954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-285eb383-e853-469f-8fda-51717a026f78 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:04.879408 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-285eb383-e853-469f-8fda-51717a026f78 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:04.879833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-285eb383-e853-469f-8fda-51717a026f78 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:04.915206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-285eb383-e853-469f-8fda-51717a026f78 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052414 Sep 08 14:06:04.915795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1219/2265] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:04 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:05.738264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4512668c-a559-4828-80fc-70dd868b6fab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:05.739180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4512668c-a559-4828-80fc-70dd868b6fab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:05.747614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4512668c-a559-4828-80fc-70dd868b6fab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:05.748110 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4512668c-a559-4828-80fc-70dd868b6fab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:05.748557 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4512668c-a559-4828-80fc-70dd868b6fab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:05.749401 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4512668c-a559-4828-80fc-70dd868b6fab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:05.749793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4512668c-a559-4828-80fc-70dd868b6fab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:05.750741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4512668c-a559-4828-80fc-70dd868b6fab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:05.902075 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4512668c-a559-4828-80fc-70dd868b6fab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:05.910798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4512668c-a559-4828-80fc-70dd868b6fab tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.176381 Sep 08 14:06:05.911561 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1220/2266] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:05 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:05.916475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e8d40787-4398-4409-a08d-4dd7f8ccb753 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:05.917177 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e8d40787-4398-4409-a08d-4dd7f8ccb753 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:05.924472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e8d40787-4398-4409-a08d-4dd7f8ccb753 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:05.925041 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e8d40787-4398-4409-a08d-4dd7f8ccb753 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:05.925596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e8d40787-4398-4409-a08d-4dd7f8ccb753 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:05.926627 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e8d40787-4398-4409-a08d-4dd7f8ccb753 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:05.927224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e8d40787-4398-4409-a08d-4dd7f8ccb753 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:05.927808 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e8d40787-4398-4409-a08d-4dd7f8ccb753 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:06.013182 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e8d40787-4398-4409-a08d-4dd7f8ccb753 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:06.024412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e8d40787-4398-4409-a08d-4dd7f8ccb753 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.111747 Sep 08 14:06:06.025000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1221/2267] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:05 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:06.029745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e239d04e-4a41-4533-aef6-94f5093306b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:06.031916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e239d04e-4a41-4533-aef6-94f5093306b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:06.038669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e239d04e-4a41-4533-aef6-94f5093306b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:06.039151 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e239d04e-4a41-4533-aef6-94f5093306b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:06.039826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e239d04e-4a41-4533-aef6-94f5093306b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:06.040316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e239d04e-4a41-4533-aef6-94f5093306b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:06.040774 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e239d04e-4a41-4533-aef6-94f5093306b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:06.041295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e239d04e-4a41-4533-aef6-94f5093306b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:06.075446 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e239d04e-4a41-4533-aef6-94f5093306b6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049372 Sep 08 14:06:06.076067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1222/2268] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:06 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:06.080992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-70ceec36-c411-4212-aaf7-0f6dcf88f318 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:06.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-70ceec36-c411-4212-aaf7-0f6dcf88f318 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:06.092517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70ceec36-c411-4212-aaf7-0f6dcf88f318 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:06.093001 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70ceec36-c411-4212-aaf7-0f6dcf88f318 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:06.093445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70ceec36-c411-4212-aaf7-0f6dcf88f318 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:06.094168 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70ceec36-c411-4212-aaf7-0f6dcf88f318 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:06.094648 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70ceec36-c411-4212-aaf7-0f6dcf88f318 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:06.095074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70ceec36-c411-4212-aaf7-0f6dcf88f318 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:06.128859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-70ceec36-c411-4212-aaf7-0f6dcf88f318 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051718 Sep 08 14:06:06.129437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1223/2269] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:06 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:06.937157 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4849c75e-f24e-4e4a-a27a-c6fc02cf8b07 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:06.937817 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4849c75e-f24e-4e4a-a27a-c6fc02cf8b07 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:06.944278 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4849c75e-f24e-4e4a-a27a-c6fc02cf8b07 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:06.944776 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4849c75e-f24e-4e4a-a27a-c6fc02cf8b07 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:06.945217 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4849c75e-f24e-4e4a-a27a-c6fc02cf8b07 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:06.945914 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4849c75e-f24e-4e4a-a27a-c6fc02cf8b07 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:06.946475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4849c75e-f24e-4e4a-a27a-c6fc02cf8b07 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:06.946907 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4849c75e-f24e-4e4a-a27a-c6fc02cf8b07 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:07.087051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4849c75e-f24e-4e4a-a27a-c6fc02cf8b07 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:07.095274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4849c75e-f24e-4e4a-a27a-c6fc02cf8b07 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.162187 Sep 08 14:06:07.095863 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1224/2270] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:06 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:07.101107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-22d3615f-3f95-4251-9b8c-2a9496314bbc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:07.101712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-22d3615f-3f95-4251-9b8c-2a9496314bbc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:07.107782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22d3615f-3f95-4251-9b8c-2a9496314bbc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:07.108268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22d3615f-3f95-4251-9b8c-2a9496314bbc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:07.108698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22d3615f-3f95-4251-9b8c-2a9496314bbc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:07.109394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22d3615f-3f95-4251-9b8c-2a9496314bbc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:07.109849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22d3615f-3f95-4251-9b8c-2a9496314bbc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:07.110321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22d3615f-3f95-4251-9b8c-2a9496314bbc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:07.192055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-22d3615f-3f95-4251-9b8c-2a9496314bbc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:07.200353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-22d3615f-3f95-4251-9b8c-2a9496314bbc tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.103471 Sep 08 14:06:07.200910 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1225/2271] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:07 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:07.205503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7f483911-9a9f-4d5b-8a46-6ce20be7dede tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:07.207879 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7f483911-9a9f-4d5b-8a46-6ce20be7dede tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:07.217495 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f483911-9a9f-4d5b-8a46-6ce20be7dede tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:07.217495 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f483911-9a9f-4d5b-8a46-6ce20be7dede tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:07.217495 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f483911-9a9f-4d5b-8a46-6ce20be7dede tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:07.217495 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f483911-9a9f-4d5b-8a46-6ce20be7dede tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:07.218047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f483911-9a9f-4d5b-8a46-6ce20be7dede tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:07.218047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7f483911-9a9f-4d5b-8a46-6ce20be7dede tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:07.234229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:07.251965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7f483911-9a9f-4d5b-8a46-6ce20be7dede tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049891 Sep 08 14:06:07.252493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1226/2272] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:07 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:07.257249 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2982ca9c-3eb1-41f3-9c6a-86df3f39439c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:07.259595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2982ca9c-3eb1-41f3-9c6a-86df3f39439c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:07.265588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2982ca9c-3eb1-41f3-9c6a-86df3f39439c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:07.266080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2982ca9c-3eb1-41f3-9c6a-86df3f39439c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:07.267271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2982ca9c-3eb1-41f3-9c6a-86df3f39439c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:07.267829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2982ca9c-3eb1-41f3-9c6a-86df3f39439c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:07.267829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2982ca9c-3eb1-41f3-9c6a-86df3f39439c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:07.268248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2982ca9c-3eb1-41f3-9c6a-86df3f39439c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:07.299405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2982ca9c-3eb1-41f3-9c6a-86df3f39439c tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045854 Sep 08 14:06:07.299976 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1227/2273] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:07 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:08.120883 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0af9072c-33de-456c-b278-a8f5138e7d71 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:08.121542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0af9072c-33de-456c-b278-a8f5138e7d71 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:08.128095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0af9072c-33de-456c-b278-a8f5138e7d71 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:08.128581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0af9072c-33de-456c-b278-a8f5138e7d71 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:08.129015 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0af9072c-33de-456c-b278-a8f5138e7d71 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:08.129704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0af9072c-33de-456c-b278-a8f5138e7d71 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:08.130291 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0af9072c-33de-456c-b278-a8f5138e7d71 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:08.131656 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0af9072c-33de-456c-b278-a8f5138e7d71 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:08.277054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0af9072c-33de-456c-b278-a8f5138e7d71 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:08.286069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0af9072c-33de-456c-b278-a8f5138e7d71 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.169427 Sep 08 14:06:08.286918 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1228/2274] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:08 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:08.291565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-55b5f33d-aacd-4579-9f86-5a44f4b20b45 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:08.292166 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-55b5f33d-aacd-4579-9f86-5a44f4b20b45 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:08.298390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-55b5f33d-aacd-4579-9f86-5a44f4b20b45 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:08.298870 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-55b5f33d-aacd-4579-9f86-5a44f4b20b45 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:08.299302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-55b5f33d-aacd-4579-9f86-5a44f4b20b45 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:08.299994 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-55b5f33d-aacd-4579-9f86-5a44f4b20b45 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:08.300448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-55b5f33d-aacd-4579-9f86-5a44f4b20b45 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:08.300865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-55b5f33d-aacd-4579-9f86-5a44f4b20b45 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:08.368016 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-55b5f33d-aacd-4579-9f86-5a44f4b20b45 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:08.376382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-55b5f33d-aacd-4579-9f86-5a44f4b20b45 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.088480 Sep 08 14:06:08.377010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1229/2275] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:08 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:08.382607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a244aab8-a0ec-46ea-96a9-3b86b16a9e1f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:08.384781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a244aab8-a0ec-46ea-96a9-3b86b16a9e1f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:08.391817 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a244aab8-a0ec-46ea-96a9-3b86b16a9e1f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:08.392087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a244aab8-a0ec-46ea-96a9-3b86b16a9e1f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:08.392440 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a244aab8-a0ec-46ea-96a9-3b86b16a9e1f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:08.393155 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a244aab8-a0ec-46ea-96a9-3b86b16a9e1f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:08.393617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a244aab8-a0ec-46ea-96a9-3b86b16a9e1f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:08.394052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a244aab8-a0ec-46ea-96a9-3b86b16a9e1f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:08.426085 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a244aab8-a0ec-46ea-96a9-3b86b16a9e1f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047908 Sep 08 14:06:08.426698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1230/2276] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:08 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:08.431850 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d98ed59d-ce66-4521-9c30-634aecd0c198 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:08.433969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d98ed59d-ce66-4521-9c30-634aecd0c198 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:08.441473 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d98ed59d-ce66-4521-9c30-634aecd0c198 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:08.442009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d98ed59d-ce66-4521-9c30-634aecd0c198 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:08.442529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d98ed59d-ce66-4521-9c30-634aecd0c198 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:08.443855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d98ed59d-ce66-4521-9c30-634aecd0c198 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:08.444343 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d98ed59d-ce66-4521-9c30-634aecd0c198 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:08.444775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d98ed59d-ce66-4521-9c30-634aecd0c198 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:08.475039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d98ed59d-ce66-4521-9c30-634aecd0c198 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046683 Sep 08 14:06:08.475616 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1231/2277] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:08 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:09.309179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-037b833c-9c2f-42d8-9486-6c92267c013f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:09.310178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-037b833c-9c2f-42d8-9486-6c92267c013f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:09.316762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-037b833c-9c2f-42d8-9486-6c92267c013f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:09.317245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-037b833c-9c2f-42d8-9486-6c92267c013f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:09.317682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-037b833c-9c2f-42d8-9486-6c92267c013f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:09.318982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-037b833c-9c2f-42d8-9486-6c92267c013f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:09.319441 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-037b833c-9c2f-42d8-9486-6c92267c013f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:09.319871 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-037b833c-9c2f-42d8-9486-6c92267c013f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:09.467396 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-037b833c-9c2f-42d8-9486-6c92267c013f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:09.480103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-037b833c-9c2f-42d8-9486-6c92267c013f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.174848 Sep 08 14:06:09.480690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1232/2278] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:09 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:09.485573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-412a9774-a674-4865-98d1-c6ecef520804 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:09.486204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-412a9774-a674-4865-98d1-c6ecef520804 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:09.492818 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-412a9774-a674-4865-98d1-c6ecef520804 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:09.493309 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-412a9774-a674-4865-98d1-c6ecef520804 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:09.493739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-412a9774-a674-4865-98d1-c6ecef520804 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:09.494465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-412a9774-a674-4865-98d1-c6ecef520804 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:09.494919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-412a9774-a674-4865-98d1-c6ecef520804 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:09.495354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-412a9774-a674-4865-98d1-c6ecef520804 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:09.565092 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-412a9774-a674-4865-98d1-c6ecef520804 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:09.576727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-412a9774-a674-4865-98d1-c6ecef520804 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.094959 Sep 08 14:06:09.577308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1233/2279] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:09 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:09.582527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3ce9b526-d925-4c28-9aac-5ed342654655 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:09.584714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3ce9b526-d925-4c28-9aac-5ed342654655 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:09.592117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3ce9b526-d925-4c28-9aac-5ed342654655 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:09.592603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3ce9b526-d925-4c28-9aac-5ed342654655 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:09.593192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3ce9b526-d925-4c28-9aac-5ed342654655 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:09.593925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3ce9b526-d925-4c28-9aac-5ed342654655 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:09.594429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3ce9b526-d925-4c28-9aac-5ed342654655 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:09.594852 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3ce9b526-d925-4c28-9aac-5ed342654655 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:09.626663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3ce9b526-d925-4c28-9aac-5ed342654655 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048254 Sep 08 14:06:09.627230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1234/2280] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:09 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:09.632068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8da79e3b-3644-4190-88b2-0b5fdd191ae7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:09.634857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8da79e3b-3644-4190-88b2-0b5fdd191ae7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:09.641433 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8da79e3b-3644-4190-88b2-0b5fdd191ae7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:09.641922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8da79e3b-3644-4190-88b2-0b5fdd191ae7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:09.642386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8da79e3b-3644-4190-88b2-0b5fdd191ae7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:09.643081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8da79e3b-3644-4190-88b2-0b5fdd191ae7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:09.643528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8da79e3b-3644-4190-88b2-0b5fdd191ae7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:09.643947 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8da79e3b-3644-4190-88b2-0b5fdd191ae7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:09.679674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8da79e3b-3644-4190-88b2-0b5fdd191ae7 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048089 Sep 08 14:06:09.679674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1235/2281] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:09 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:10.504494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0f32ed90-6860-4f1e-8478-2143275caae4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:10.505204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0f32ed90-6860-4f1e-8478-2143275caae4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:10.512782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0f32ed90-6860-4f1e-8478-2143275caae4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:10.513263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0f32ed90-6860-4f1e-8478-2143275caae4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:10.513695 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0f32ed90-6860-4f1e-8478-2143275caae4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:10.514431 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0f32ed90-6860-4f1e-8478-2143275caae4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:10.514883 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0f32ed90-6860-4f1e-8478-2143275caae4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:10.515836 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0f32ed90-6860-4f1e-8478-2143275caae4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:10.662728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0f32ed90-6860-4f1e-8478-2143275caae4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:10.675340 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0f32ed90-6860-4f1e-8478-2143275caae4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.174869 Sep 08 14:06:10.675935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1236/2282] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:10 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:10.681248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3191e54f-c105-4da7-89cb-e51e0159f038 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:10.681966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3191e54f-c105-4da7-89cb-e51e0159f038 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:10.688490 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3191e54f-c105-4da7-89cb-e51e0159f038 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:10.688982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3191e54f-c105-4da7-89cb-e51e0159f038 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:10.689452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3191e54f-c105-4da7-89cb-e51e0159f038 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:10.690252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3191e54f-c105-4da7-89cb-e51e0159f038 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:10.690646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3191e54f-c105-4da7-89cb-e51e0159f038 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:10.691075 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3191e54f-c105-4da7-89cb-e51e0159f038 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:10.760079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3191e54f-c105-4da7-89cb-e51e0159f038 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:10.768630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3191e54f-c105-4da7-89cb-e51e0159f038 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.091629 Sep 08 14:06:10.769226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1237/2283] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:10 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:10.780031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-576574a3-016b-46fa-875c-7a8d0dfdb0da tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:10.782251 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-576574a3-016b-46fa-875c-7a8d0dfdb0da tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:10.789468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-576574a3-016b-46fa-875c-7a8d0dfdb0da tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:10.789958 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-576574a3-016b-46fa-875c-7a8d0dfdb0da tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:10.790505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-576574a3-016b-46fa-875c-7a8d0dfdb0da tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:10.791216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-576574a3-016b-46fa-875c-7a8d0dfdb0da tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:10.791668 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-576574a3-016b-46fa-875c-7a8d0dfdb0da tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:10.792093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-576574a3-016b-46fa-875c-7a8d0dfdb0da tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:10.827370 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-576574a3-016b-46fa-875c-7a8d0dfdb0da tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057064 Sep 08 14:06:10.827951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1238/2284] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:10 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:10.832837 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b09b028c-ec14-42b3-a8c9-9198874ce3a8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:10.835014 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b09b028c-ec14-42b3-a8c9-9198874ce3a8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:10.841177 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b09b028c-ec14-42b3-a8c9-9198874ce3a8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:10.841661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b09b028c-ec14-42b3-a8c9-9198874ce3a8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:10.842098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b09b028c-ec14-42b3-a8c9-9198874ce3a8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:10.842828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b09b028c-ec14-42b3-a8c9-9198874ce3a8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:10.843282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b09b028c-ec14-42b3-a8c9-9198874ce3a8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:10.843703 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b09b028c-ec14-42b3-a8c9-9198874ce3a8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:10.881123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b09b028c-ec14-42b3-a8c9-9198874ce3a8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052026 Sep 08 14:06:10.881710 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1239/2285] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:10 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:11.702359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a60d90dd-390d-4444-8ab7-a5e42eb80712 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:11.703053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a60d90dd-390d-4444-8ab7-a5e42eb80712 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:11.705594 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:11.711490 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a60d90dd-390d-4444-8ab7-a5e42eb80712 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:11.711983 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a60d90dd-390d-4444-8ab7-a5e42eb80712 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:11.712427 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a60d90dd-390d-4444-8ab7-a5e42eb80712 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:11.713139 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a60d90dd-390d-4444-8ab7-a5e42eb80712 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:11.713604 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a60d90dd-390d-4444-8ab7-a5e42eb80712 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:11.714112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a60d90dd-390d-4444-8ab7-a5e42eb80712 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:11.879328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a60d90dd-390d-4444-8ab7-a5e42eb80712 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:11.888202 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a60d90dd-390d-4444-8ab7-a5e42eb80712 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.189536 Sep 08 14:06:11.888822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1240/2286] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:11 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:11.894334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-014ba4f5-3312-461e-9f21-33e9408ebbcd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:11.894846 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-014ba4f5-3312-461e-9f21-33e9408ebbcd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:11.902566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-014ba4f5-3312-461e-9f21-33e9408ebbcd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:11.903160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-014ba4f5-3312-461e-9f21-33e9408ebbcd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:11.903698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-014ba4f5-3312-461e-9f21-33e9408ebbcd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:11.904480 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-014ba4f5-3312-461e-9f21-33e9408ebbcd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:11.905093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-014ba4f5-3312-461e-9f21-33e9408ebbcd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:11.905622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-014ba4f5-3312-461e-9f21-33e9408ebbcd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:11.984048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-014ba4f5-3312-461e-9f21-33e9408ebbcd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:11.994677 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-014ba4f5-3312-461e-9f21-33e9408ebbcd tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.104711 Sep 08 14:06:11.995257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1241/2287] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:11 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:11.999949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4fd777fe-d8c2-4eab-b731-5c39dbc5cd82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:12.002108 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4fd777fe-d8c2-4eab-b731-5c39dbc5cd82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:12.008515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fd777fe-d8c2-4eab-b731-5c39dbc5cd82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:12.009013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fd777fe-d8c2-4eab-b731-5c39dbc5cd82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:12.009456 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fd777fe-d8c2-4eab-b731-5c39dbc5cd82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:12.010189 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fd777fe-d8c2-4eab-b731-5c39dbc5cd82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:12.010655 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fd777fe-d8c2-4eab-b731-5c39dbc5cd82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:12.011087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fd777fe-d8c2-4eab-b731-5c39dbc5cd82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:12.044972 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4fd777fe-d8c2-4eab-b731-5c39dbc5cd82 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048638 Sep 08 14:06:12.045552 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1242/2288] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:11 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:12.051143 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ef7edeee-dcf8-432a-a136-1676ac88a937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:12.053386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ef7edeee-dcf8-432a-a136-1676ac88a937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:12.059588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef7edeee-dcf8-432a-a136-1676ac88a937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:12.060080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef7edeee-dcf8-432a-a136-1676ac88a937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:12.060526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef7edeee-dcf8-432a-a136-1676ac88a937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:12.061231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef7edeee-dcf8-432a-a136-1676ac88a937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:12.061686 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef7edeee-dcf8-432a-a136-1676ac88a937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:12.062118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef7edeee-dcf8-432a-a136-1676ac88a937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:12.064686 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:12.093889 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ef7edeee-dcf8-432a-a136-1676ac88a937 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047266 Sep 08 14:06:12.094500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1243/2289] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:12 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:12.913825 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-68d97975-84e1-4ff0-9011-09eabe6e18b6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:12.914541 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-68d97975-84e1-4ff0-9011-09eabe6e18b6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:12.921568 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68d97975-84e1-4ff0-9011-09eabe6e18b6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:12.922093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68d97975-84e1-4ff0-9011-09eabe6e18b6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:12.922531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68d97975-84e1-4ff0-9011-09eabe6e18b6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:12.923226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68d97975-84e1-4ff0-9011-09eabe6e18b6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:12.923733 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68d97975-84e1-4ff0-9011-09eabe6e18b6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:12.924124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68d97975-84e1-4ff0-9011-09eabe6e18b6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:13.073145 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-68d97975-84e1-4ff0-9011-09eabe6e18b6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:13.081757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-68d97975-84e1-4ff0-9011-09eabe6e18b6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.172608 Sep 08 14:06:13.082387 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1244/2290] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:12 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:13.087603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f2d551b0-e399-429e-bb3c-1c63ab4dedb7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:13.088213 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f2d551b0-e399-429e-bb3c-1c63ab4dedb7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:13.094249 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2d551b0-e399-429e-bb3c-1c63ab4dedb7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:13.094742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2d551b0-e399-429e-bb3c-1c63ab4dedb7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:13.095184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2d551b0-e399-429e-bb3c-1c63ab4dedb7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:13.095888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2d551b0-e399-429e-bb3c-1c63ab4dedb7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:13.096347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2d551b0-e399-429e-bb3c-1c63ab4dedb7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:13.096777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2d551b0-e399-429e-bb3c-1c63ab4dedb7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:13.162256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f2d551b0-e399-429e-bb3c-1c63ab4dedb7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:13.172601 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f2d551b0-e399-429e-bb3c-1c63ab4dedb7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.089070 Sep 08 14:06:13.173112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1245/2291] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:13 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:13.177922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bbd884aa-411a-4ce2-99b2-30cfd6c0617f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:13.180164 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bbd884aa-411a-4ce2-99b2-30cfd6c0617f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:13.192977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bbd884aa-411a-4ce2-99b2-30cfd6c0617f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:13.193479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bbd884aa-411a-4ce2-99b2-30cfd6c0617f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:13.193931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bbd884aa-411a-4ce2-99b2-30cfd6c0617f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:13.195192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bbd884aa-411a-4ce2-99b2-30cfd6c0617f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:13.195715 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bbd884aa-411a-4ce2-99b2-30cfd6c0617f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:13.196151 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bbd884aa-411a-4ce2-99b2-30cfd6c0617f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:13.227822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bbd884aa-411a-4ce2-99b2-30cfd6c0617f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053656 Sep 08 14:06:13.228388 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1246/2292] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:13 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:13.234409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b21d9bf4-3eff-49eb-bf95-a31e675a9b20 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:13.236579 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b21d9bf4-3eff-49eb-bf95-a31e675a9b20 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:13.242835 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b21d9bf4-3eff-49eb-bf95-a31e675a9b20 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:13.243327 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b21d9bf4-3eff-49eb-bf95-a31e675a9b20 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:13.243766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b21d9bf4-3eff-49eb-bf95-a31e675a9b20 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:13.244461 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b21d9bf4-3eff-49eb-bf95-a31e675a9b20 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:13.244919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b21d9bf4-3eff-49eb-bf95-a31e675a9b20 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:13.245348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b21d9bf4-3eff-49eb-bf95-a31e675a9b20 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:13.278540 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b21d9bf4-3eff-49eb-bf95-a31e675a9b20 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048867 Sep 08 14:06:13.279136 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1247/2293] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:13 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:14.109350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7b4f6628-a96a-4bf2-8bb0-cfbe7d72713b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:14.110198 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7b4f6628-a96a-4bf2-8bb0-cfbe7d72713b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:14.118482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b4f6628-a96a-4bf2-8bb0-cfbe7d72713b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:14.119014 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b4f6628-a96a-4bf2-8bb0-cfbe7d72713b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:14.119458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b4f6628-a96a-4bf2-8bb0-cfbe7d72713b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:14.120171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b4f6628-a96a-4bf2-8bb0-cfbe7d72713b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:14.120654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b4f6628-a96a-4bf2-8bb0-cfbe7d72713b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:14.121085 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7b4f6628-a96a-4bf2-8bb0-cfbe7d72713b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:14.283366 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7b4f6628-a96a-4bf2-8bb0-cfbe7d72713b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:14.292094 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7b4f6628-a96a-4bf2-8bb0-cfbe7d72713b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.186775 Sep 08 14:06:14.292674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1248/2294] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:14 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:14.298386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-43f3199a-9b53-4399-9fcf-07db044fef4e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:14.298807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-43f3199a-9b53-4399-9fcf-07db044fef4e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:14.305740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43f3199a-9b53-4399-9fcf-07db044fef4e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:14.306330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43f3199a-9b53-4399-9fcf-07db044fef4e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:14.306771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43f3199a-9b53-4399-9fcf-07db044fef4e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:14.307473 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43f3199a-9b53-4399-9fcf-07db044fef4e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:14.307927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43f3199a-9b53-4399-9fcf-07db044fef4e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:14.308362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43f3199a-9b53-4399-9fcf-07db044fef4e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:14.399093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-43f3199a-9b53-4399-9fcf-07db044fef4e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:14.413650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-43f3199a-9b53-4399-9fcf-07db044fef4e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.119059 Sep 08 14:06:14.414733 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1249/2295] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:14 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:14.421104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8cefcb91-a3d5-4153-b8fb-79dce2d378fb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:14.423440 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8cefcb91-a3d5-4153-b8fb-79dce2d378fb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:14.430231 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8cefcb91-a3d5-4153-b8fb-79dce2d378fb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:14.430723 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8cefcb91-a3d5-4153-b8fb-79dce2d378fb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:14.431162 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8cefcb91-a3d5-4153-b8fb-79dce2d378fb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:14.431874 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8cefcb91-a3d5-4153-b8fb-79dce2d378fb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:14.432328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8cefcb91-a3d5-4153-b8fb-79dce2d378fb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:14.432749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8cefcb91-a3d5-4153-b8fb-79dce2d378fb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:14.467306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8cefcb91-a3d5-4153-b8fb-79dce2d378fb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050928 Sep 08 14:06:14.467877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1250/2296] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:14 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:14.473892 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fca4ede4-6b9e-4614-b357-faca28efbb14 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:14.478122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fca4ede4-6b9e-4614-b357-faca28efbb14 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:14.486290 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fca4ede4-6b9e-4614-b357-faca28efbb14 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:14.486880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fca4ede4-6b9e-4614-b357-faca28efbb14 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:14.487363 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fca4ede4-6b9e-4614-b357-faca28efbb14 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:14.488211 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fca4ede4-6b9e-4614-b357-faca28efbb14 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:14.488811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fca4ede4-6b9e-4614-b357-faca28efbb14 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:14.489303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fca4ede4-6b9e-4614-b357-faca28efbb14 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:14.521863 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fca4ede4-6b9e-4614-b357-faca28efbb14 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052921 Sep 08 14:06:14.522520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1251/2297] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:14 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:15.318645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-971e7747-c255-4717-80c4-fd9c463ccf44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:15.320104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-971e7747-c255-4717-80c4-fd9c463ccf44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:15.326941 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-971e7747-c255-4717-80c4-fd9c463ccf44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:15.327433 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-971e7747-c255-4717-80c4-fd9c463ccf44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:15.327869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-971e7747-c255-4717-80c4-fd9c463ccf44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:15.328608 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-971e7747-c255-4717-80c4-fd9c463ccf44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:15.329057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-971e7747-c255-4717-80c4-fd9c463ccf44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:15.329477 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-971e7747-c255-4717-80c4-fd9c463ccf44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:15.523641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-971e7747-c255-4717-80c4-fd9c463ccf44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:15.532987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-971e7747-c255-4717-80c4-fd9c463ccf44 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.218790 Sep 08 14:06:15.533649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1252/2298] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:15 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:15.542943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-97fa0fbb-3263-45e5-a1c1-6db7f0f79278 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:15.544099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-97fa0fbb-3263-45e5-a1c1-6db7f0f79278 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:15.556190 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-97fa0fbb-3263-45e5-a1c1-6db7f0f79278 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:15.557149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-97fa0fbb-3263-45e5-a1c1-6db7f0f79278 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:15.557922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-97fa0fbb-3263-45e5-a1c1-6db7f0f79278 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:15.559282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-97fa0fbb-3263-45e5-a1c1-6db7f0f79278 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:15.560201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-97fa0fbb-3263-45e5-a1c1-6db7f0f79278 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:15.561006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-97fa0fbb-3263-45e5-a1c1-6db7f0f79278 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:15.670650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-97fa0fbb-3263-45e5-a1c1-6db7f0f79278 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:15.679448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-97fa0fbb-3263-45e5-a1c1-6db7f0f79278 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.144303 Sep 08 14:06:15.680043 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1253/2299] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:15 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:15.685146 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1d126577-6e7c-4875-a8b8-0d52aaebf44d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:15.688520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1d126577-6e7c-4875-a8b8-0d52aaebf44d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:15.694723 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d126577-6e7c-4875-a8b8-0d52aaebf44d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:15.695218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d126577-6e7c-4875-a8b8-0d52aaebf44d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:15.695713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d126577-6e7c-4875-a8b8-0d52aaebf44d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:15.696599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d126577-6e7c-4875-a8b8-0d52aaebf44d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:15.697062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d126577-6e7c-4875-a8b8-0d52aaebf44d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:15.697491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d126577-6e7c-4875-a8b8-0d52aaebf44d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:15.733910 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1d126577-6e7c-4875-a8b8-0d52aaebf44d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052739 Sep 08 14:06:15.734530 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1254/2300] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:15 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:15.739450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-91a9db80-3840-4452-afab-59a6faeb52e2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:15.742114 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-91a9db80-3840-4452-afab-59a6faeb52e2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:15.748960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91a9db80-3840-4452-afab-59a6faeb52e2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:15.749458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91a9db80-3840-4452-afab-59a6faeb52e2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:15.749906 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91a9db80-3840-4452-afab-59a6faeb52e2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:15.750684 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91a9db80-3840-4452-afab-59a6faeb52e2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:15.751235 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91a9db80-3840-4452-afab-59a6faeb52e2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:15.751809 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91a9db80-3840-4452-afab-59a6faeb52e2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:15.788421 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-91a9db80-3840-4452-afab-59a6faeb52e2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052159 Sep 08 14:06:15.789018 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1255/2301] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:15 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:16.577030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0257cbcb-48f0-4868-830f-db105d764548 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:16.577895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0257cbcb-48f0-4868-830f-db105d764548 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:16.586116 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0257cbcb-48f0-4868-830f-db105d764548 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:16.589748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0257cbcb-48f0-4868-830f-db105d764548 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:16.589748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0257cbcb-48f0-4868-830f-db105d764548 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:16.589748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0257cbcb-48f0-4868-830f-db105d764548 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:16.589748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0257cbcb-48f0-4868-830f-db105d764548 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:16.590197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0257cbcb-48f0-4868-830f-db105d764548 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:16.752969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0257cbcb-48f0-4868-830f-db105d764548 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:16.761035 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0257cbcb-48f0-4868-830f-db105d764548 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.189918 Sep 08 14:06:16.761612 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1256/2302] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:16 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:16.766650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8d9bdc2b-7508-4f9f-9378-f47e6f7d2110 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:16.767586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8d9bdc2b-7508-4f9f-9378-f47e6f7d2110 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:16.774373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8d9bdc2b-7508-4f9f-9378-f47e6f7d2110 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:16.776291 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8d9bdc2b-7508-4f9f-9378-f47e6f7d2110 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:16.776889 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8d9bdc2b-7508-4f9f-9378-f47e6f7d2110 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:16.777706 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8d9bdc2b-7508-4f9f-9378-f47e6f7d2110 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:16.778191 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8d9bdc2b-7508-4f9f-9378-f47e6f7d2110 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:16.778624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8d9bdc2b-7508-4f9f-9378-f47e6f7d2110 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:16.852072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8d9bdc2b-7508-4f9f-9378-f47e6f7d2110 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:16.860178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8d9bdc2b-7508-4f9f-9378-f47e6f7d2110 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.097549 Sep 08 14:06:16.860765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1257/2303] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:16 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:16.866724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-109f2d38-0600-42f8-8ad8-d4b3b8e40428 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:16.868887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-109f2d38-0600-42f8-8ad8-d4b3b8e40428 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:16.875064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-109f2d38-0600-42f8-8ad8-d4b3b8e40428 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:16.875729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-109f2d38-0600-42f8-8ad8-d4b3b8e40428 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:16.876333 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-109f2d38-0600-42f8-8ad8-d4b3b8e40428 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:16.876828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-109f2d38-0600-42f8-8ad8-d4b3b8e40428 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:16.877290 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-109f2d38-0600-42f8-8ad8-d4b3b8e40428 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:16.877734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-109f2d38-0600-42f8-8ad8-d4b3b8e40428 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:16.910635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-109f2d38-0600-42f8-8ad8-d4b3b8e40428 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048757 Sep 08 14:06:16.911220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1258/2304] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:16 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:16.916399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3fc9efa1-af8a-400f-a048-d74f43e393a4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:16.918583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3fc9efa1-af8a-400f-a048-d74f43e393a4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:16.924942 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3fc9efa1-af8a-400f-a048-d74f43e393a4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:16.925442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3fc9efa1-af8a-400f-a048-d74f43e393a4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:16.925943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3fc9efa1-af8a-400f-a048-d74f43e393a4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:16.926683 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3fc9efa1-af8a-400f-a048-d74f43e393a4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:16.927139 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3fc9efa1-af8a-400f-a048-d74f43e393a4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:16.927569 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3fc9efa1-af8a-400f-a048-d74f43e393a4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:16.954434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:16.960964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3fc9efa1-af8a-400f-a048-d74f43e393a4 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048749 Sep 08 14:06:16.961574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1259/2305] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:16 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:17.791415 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-463a154d-f887-4467-877c-49bbb0614579 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:17.792387 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-463a154d-f887-4467-877c-49bbb0614579 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:17.799951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-463a154d-f887-4467-877c-49bbb0614579 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:17.800450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-463a154d-f887-4467-877c-49bbb0614579 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:17.800895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-463a154d-f887-4467-877c-49bbb0614579 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:17.801634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-463a154d-f887-4467-877c-49bbb0614579 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:17.802095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-463a154d-f887-4467-877c-49bbb0614579 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:17.802565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-463a154d-f887-4467-877c-49bbb0614579 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:17.959578 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-463a154d-f887-4467-877c-49bbb0614579 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:17.971072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-463a154d-f887-4467-877c-49bbb0614579 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.184030 Sep 08 14:06:17.971873 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1260/2306] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:17 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:17.979323 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-256c9da7-3b6d-4f35-a159-681bc2177945 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:17.980232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-256c9da7-3b6d-4f35-a159-681bc2177945 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:17.989681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-256c9da7-3b6d-4f35-a159-681bc2177945 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:17.990570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-256c9da7-3b6d-4f35-a159-681bc2177945 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:17.991590 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-256c9da7-3b6d-4f35-a159-681bc2177945 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:17.992727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-256c9da7-3b6d-4f35-a159-681bc2177945 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:17.993196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-256c9da7-3b6d-4f35-a159-681bc2177945 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:17.993998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-256c9da7-3b6d-4f35-a159-681bc2177945 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:18.079535 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-256c9da7-3b6d-4f35-a159-681bc2177945 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:18.097021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-256c9da7-3b6d-4f35-a159-681bc2177945 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.123079 Sep 08 14:06:18.098078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1261/2307] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:17 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:18.107196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b1611680-62fe-4530-a0b1-d76d0d8bb788 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:18.110969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b1611680-62fe-4530-a0b1-d76d0d8bb788 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:18.122014 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b1611680-62fe-4530-a0b1-d76d0d8bb788 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:18.122944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b1611680-62fe-4530-a0b1-d76d0d8bb788 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:18.123708 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b1611680-62fe-4530-a0b1-d76d0d8bb788 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:18.124505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b1611680-62fe-4530-a0b1-d76d0d8bb788 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:18.125423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b1611680-62fe-4530-a0b1-d76d0d8bb788 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:18.126187 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b1611680-62fe-4530-a0b1-d76d0d8bb788 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:18.164129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b1611680-62fe-4530-a0b1-d76d0d8bb788 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064351 Sep 08 14:06:18.164741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1262/2308] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:18 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:18.170935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b7109c80-27b3-4f5d-ac2e-e3fc42e133aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:18.173668 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b7109c80-27b3-4f5d-ac2e-e3fc42e133aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:18.181816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7109c80-27b3-4f5d-ac2e-e3fc42e133aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:18.182659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7109c80-27b3-4f5d-ac2e-e3fc42e133aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:18.183359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7109c80-27b3-4f5d-ac2e-e3fc42e133aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:18.184903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7109c80-27b3-4f5d-ac2e-e3fc42e133aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:18.185680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7109c80-27b3-4f5d-ac2e-e3fc42e133aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:18.186466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7109c80-27b3-4f5d-ac2e-e3fc42e133aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:18.234493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b7109c80-27b3-4f5d-ac2e-e3fc42e133aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.068524 Sep 08 14:06:18.235068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1263/2309] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:18 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:19.002494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b2bdeb70-7f20-4462-9dbc-d7cfef4930a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:19.003445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b2bdeb70-7f20-4462-9dbc-d7cfef4930a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:19.013272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b2bdeb70-7f20-4462-9dbc-d7cfef4930a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:19.013834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b2bdeb70-7f20-4462-9dbc-d7cfef4930a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:19.014322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b2bdeb70-7f20-4462-9dbc-d7cfef4930a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:19.015237 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b2bdeb70-7f20-4462-9dbc-d7cfef4930a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:19.015697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b2bdeb70-7f20-4462-9dbc-d7cfef4930a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:19.016126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b2bdeb70-7f20-4462-9dbc-d7cfef4930a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:19.186176 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b2bdeb70-7f20-4462-9dbc-d7cfef4930a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:19.194736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b2bdeb70-7f20-4462-9dbc-d7cfef4930a8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.197208 Sep 08 14:06:19.195345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1264/2310] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:18 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:19.199987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f0b3dfba-25c4-4c31-a5bb-dcb7d16a1d82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:19.200595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f0b3dfba-25c4-4c31-a5bb-dcb7d16a1d82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:19.207276 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0b3dfba-25c4-4c31-a5bb-dcb7d16a1d82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:19.207758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0b3dfba-25c4-4c31-a5bb-dcb7d16a1d82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:19.208197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0b3dfba-25c4-4c31-a5bb-dcb7d16a1d82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:19.209108 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0b3dfba-25c4-4c31-a5bb-dcb7d16a1d82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:19.209560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0b3dfba-25c4-4c31-a5bb-dcb7d16a1d82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:19.210077 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f0b3dfba-25c4-4c31-a5bb-dcb7d16a1d82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:19.296397 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f0b3dfba-25c4-4c31-a5bb-dcb7d16a1d82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:19.310967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f0b3dfba-25c4-4c31-a5bb-dcb7d16a1d82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.114217 Sep 08 14:06:19.311937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1265/2311] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:19 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:19.321226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b0a8933a-e708-40b3-b369-5e39ba0ac8ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:19.325196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b0a8933a-e708-40b3-b369-5e39ba0ac8ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:19.333795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b0a8933a-e708-40b3-b369-5e39ba0ac8ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:19.334450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b0a8933a-e708-40b3-b369-5e39ba0ac8ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:19.334900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b0a8933a-e708-40b3-b369-5e39ba0ac8ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:19.335615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b0a8933a-e708-40b3-b369-5e39ba0ac8ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:19.336078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b0a8933a-e708-40b3-b369-5e39ba0ac8ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:19.336508 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b0a8933a-e708-40b3-b369-5e39ba0ac8ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:19.375049 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b0a8933a-e708-40b3-b369-5e39ba0ac8ec tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061569 Sep 08 14:06:19.375665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1266/2312] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:19 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:19.381422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-31805005-386e-41bc-be16-6cddcb145c92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:19.385072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-31805005-386e-41bc-be16-6cddcb145c92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:19.396523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31805005-386e-41bc-be16-6cddcb145c92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:19.397375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31805005-386e-41bc-be16-6cddcb145c92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:19.398065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31805005-386e-41bc-be16-6cddcb145c92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:19.399205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31805005-386e-41bc-be16-6cddcb145c92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:19.399947 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31805005-386e-41bc-be16-6cddcb145c92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:19.400624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31805005-386e-41bc-be16-6cddcb145c92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:19.437678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-31805005-386e-41bc-be16-6cddcb145c92 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060876 Sep 08 14:06:19.438347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1267/2313] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:19 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:20.223794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-daff4646-1b45-43c4-9058-ed36c89dde61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:20.224547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-daff4646-1b45-43c4-9058-ed36c89dde61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:20.233378 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-daff4646-1b45-43c4-9058-ed36c89dde61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:20.233951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-daff4646-1b45-43c4-9058-ed36c89dde61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:20.234583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-daff4646-1b45-43c4-9058-ed36c89dde61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:20.235638 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-daff4646-1b45-43c4-9058-ed36c89dde61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:20.236274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-daff4646-1b45-43c4-9058-ed36c89dde61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:20.236779 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-daff4646-1b45-43c4-9058-ed36c89dde61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:20.427382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-daff4646-1b45-43c4-9058-ed36c89dde61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:20.436093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-daff4646-1b45-43c4-9058-ed36c89dde61 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.217761 Sep 08 14:06:20.436669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1268/2314] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:20 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:20.441419 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-936e4cea-e69d-4258-83f5-b6bf0b74f585 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:20.442028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-936e4cea-e69d-4258-83f5-b6bf0b74f585 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:20.450397 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-936e4cea-e69d-4258-83f5-b6bf0b74f585 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:20.450963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-936e4cea-e69d-4258-83f5-b6bf0b74f585 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:20.451337 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-936e4cea-e69d-4258-83f5-b6bf0b74f585 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:20.452075 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-936e4cea-e69d-4258-83f5-b6bf0b74f585 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:20.452497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-936e4cea-e69d-4258-83f5-b6bf0b74f585 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:20.452925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-936e4cea-e69d-4258-83f5-b6bf0b74f585 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:20.536941 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-936e4cea-e69d-4258-83f5-b6bf0b74f585 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:20.546503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-936e4cea-e69d-4258-83f5-b6bf0b74f585 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.108641 Sep 08 14:06:20.547298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1269/2315] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:20 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:20.553924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-64c2a3fc-8e3d-43d9-9a59-99c1d190ebe0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:20.556283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-64c2a3fc-8e3d-43d9-9a59-99c1d190ebe0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:20.564487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-64c2a3fc-8e3d-43d9-9a59-99c1d190ebe0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:20.565157 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-64c2a3fc-8e3d-43d9-9a59-99c1d190ebe0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:20.565613 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-64c2a3fc-8e3d-43d9-9a59-99c1d190ebe0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:20.566546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-64c2a3fc-8e3d-43d9-9a59-99c1d190ebe0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:20.566854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-64c2a3fc-8e3d-43d9-9a59-99c1d190ebe0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:20.567293 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-64c2a3fc-8e3d-43d9-9a59-99c1d190ebe0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:20.616634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-64c2a3fc-8e3d-43d9-9a59-99c1d190ebe0 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.068107 Sep 08 14:06:20.617138 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1270/2316] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:20 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:20.623448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-583cf5dc-2dc1-40e2-9988-c19606c12c3e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:20.625765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-583cf5dc-2dc1-40e2-9988-c19606c12c3e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:20.633080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-583cf5dc-2dc1-40e2-9988-c19606c12c3e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:20.634681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-583cf5dc-2dc1-40e2-9988-c19606c12c3e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:20.634681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-583cf5dc-2dc1-40e2-9988-c19606c12c3e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:20.635601 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-583cf5dc-2dc1-40e2-9988-c19606c12c3e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:20.636321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-583cf5dc-2dc1-40e2-9988-c19606c12c3e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:20.636979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-583cf5dc-2dc1-40e2-9988-c19606c12c3e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:20.692632 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-583cf5dc-2dc1-40e2-9988-c19606c12c3e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.074231 Sep 08 14:06:20.693221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1271/2317] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:20 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:21.466210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c0d45c40-0461-49b1-8e36-e0bf4093f815 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:21.467250 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c0d45c40-0461-49b1-8e36-e0bf4093f815 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:21.475528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0d45c40-0461-49b1-8e36-e0bf4093f815 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:21.476176 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0d45c40-0461-49b1-8e36-e0bf4093f815 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:21.476622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0d45c40-0461-49b1-8e36-e0bf4093f815 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:21.477349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0d45c40-0461-49b1-8e36-e0bf4093f815 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:21.477808 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0d45c40-0461-49b1-8e36-e0bf4093f815 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:21.478295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c0d45c40-0461-49b1-8e36-e0bf4093f815 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:21.665374 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c0d45c40-0461-49b1-8e36-e0bf4093f815 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:21.675039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c0d45c40-0461-49b1-8e36-e0bf4093f815 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.216906 Sep 08 14:06:21.678464 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1272/2318] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:21 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:21.680256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0d2e6d41-10ff-40f6-987c-b5d645930a6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:21.680856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0d2e6d41-10ff-40f6-987c-b5d645930a6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:21.690519 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d2e6d41-10ff-40f6-987c-b5d645930a6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:21.691021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d2e6d41-10ff-40f6-987c-b5d645930a6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:21.691591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d2e6d41-10ff-40f6-987c-b5d645930a6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:21.692465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d2e6d41-10ff-40f6-987c-b5d645930a6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:21.693184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d2e6d41-10ff-40f6-987c-b5d645930a6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:21.693956 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0d2e6d41-10ff-40f6-987c-b5d645930a6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:21.782342 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0d2e6d41-10ff-40f6-987c-b5d645930a6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:21.792126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0d2e6d41-10ff-40f6-987c-b5d645930a6c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.115393 Sep 08 14:06:21.792727 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1273/2319] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:21 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:21.797814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-26e3cd24-0d14-47b4-807d-cd449b476a4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:21.800247 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-26e3cd24-0d14-47b4-807d-cd449b476a4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:21.810684 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26e3cd24-0d14-47b4-807d-cd449b476a4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:21.811339 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26e3cd24-0d14-47b4-807d-cd449b476a4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:21.812068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26e3cd24-0d14-47b4-807d-cd449b476a4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:21.812945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26e3cd24-0d14-47b4-807d-cd449b476a4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:21.814214 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26e3cd24-0d14-47b4-807d-cd449b476a4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:21.814214 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26e3cd24-0d14-47b4-807d-cd449b476a4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:21.866521 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-26e3cd24-0d14-47b4-807d-cd449b476a4f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.071444 Sep 08 14:06:21.867559 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1274/2320] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:21 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:21.876287 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-81ec9922-19e6-438a-b645-bbbfde64fc77 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:21.879963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-81ec9922-19e6-438a-b645-bbbfde64fc77 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:21.891663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81ec9922-19e6-438a-b645-bbbfde64fc77 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:21.892555 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81ec9922-19e6-438a-b645-bbbfde64fc77 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:21.893330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81ec9922-19e6-438a-b645-bbbfde64fc77 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:21.894282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81ec9922-19e6-438a-b645-bbbfde64fc77 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:21.894943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81ec9922-19e6-438a-b645-bbbfde64fc77 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:21.895562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81ec9922-19e6-438a-b645-bbbfde64fc77 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:21.944952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-81ec9922-19e6-438a-b645-bbbfde64fc77 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.075403 Sep 08 14:06:21.945862 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1275/2321] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:21 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:22.704743 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3d9f0256-f511-4540-95cb-dc68453724cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:22.705833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3d9f0256-f511-4540-95cb-dc68453724cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:22.715434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d9f0256-f511-4540-95cb-dc68453724cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:22.716101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d9f0256-f511-4540-95cb-dc68453724cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:22.716650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d9f0256-f511-4540-95cb-dc68453724cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:22.717435 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d9f0256-f511-4540-95cb-dc68453724cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:22.717970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d9f0256-f511-4540-95cb-dc68453724cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:22.718517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d9f0256-f511-4540-95cb-dc68453724cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:22.748009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:22.920398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3d9f0256-f511-4540-95cb-dc68453724cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:22.930744 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3d9f0256-f511-4540-95cb-dc68453724cf tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.232910 Sep 08 14:06:22.931633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1276/2322] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:22 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:22.937643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a08f61a4-09c1-4e4b-be76-f640ec83a3b7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:22.938491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a08f61a4-09c1-4e4b-be76-f640ec83a3b7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:22.947345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a08f61a4-09c1-4e4b-be76-f640ec83a3b7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:22.947990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a08f61a4-09c1-4e4b-be76-f640ec83a3b7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:22.948496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a08f61a4-09c1-4e4b-be76-f640ec83a3b7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:22.950207 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a08f61a4-09c1-4e4b-be76-f640ec83a3b7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:22.950207 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a08f61a4-09c1-4e4b-be76-f640ec83a3b7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:22.950632 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a08f61a4-09c1-4e4b-be76-f640ec83a3b7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:23.036031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a08f61a4-09c1-4e4b-be76-f640ec83a3b7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:23.044972 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a08f61a4-09c1-4e4b-be76-f640ec83a3b7 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.112063 Sep 08 14:06:23.045562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1277/2323] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:22 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:23.050785 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4fb8533c-e256-4eff-ad51-df3b75f7c7b5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:23.053631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4fb8533c-e256-4eff-ad51-df3b75f7c7b5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:23.060274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fb8533c-e256-4eff-ad51-df3b75f7c7b5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:23.060787 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fb8533c-e256-4eff-ad51-df3b75f7c7b5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:23.061178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fb8533c-e256-4eff-ad51-df3b75f7c7b5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:23.061838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fb8533c-e256-4eff-ad51-df3b75f7c7b5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:23.062325 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fb8533c-e256-4eff-ad51-df3b75f7c7b5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:23.062906 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4fb8533c-e256-4eff-ad51-df3b75f7c7b5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:23.096949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4fb8533c-e256-4eff-ad51-df3b75f7c7b5 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050239 Sep 08 14:06:23.097613 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1278/2324] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:23 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:23.103584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-323db50c-1084-48aa-aaea-db0a94a9af83 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:23.105903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-323db50c-1084-48aa-aaea-db0a94a9af83 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:23.112731 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-323db50c-1084-48aa-aaea-db0a94a9af83 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:23.113267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-323db50c-1084-48aa-aaea-db0a94a9af83 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:23.113739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-323db50c-1084-48aa-aaea-db0a94a9af83 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:23.114540 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-323db50c-1084-48aa-aaea-db0a94a9af83 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:23.115322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-323db50c-1084-48aa-aaea-db0a94a9af83 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:23.116731 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-323db50c-1084-48aa-aaea-db0a94a9af83 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:23.150170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-323db50c-1084-48aa-aaea-db0a94a9af83 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051245 Sep 08 14:06:23.150776 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1279/2325] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:23 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:23.960676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2efeb480-e25c-4839-bd64-29faffc3d9da tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:23.961765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2efeb480-e25c-4839-bd64-29faffc3d9da tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:23.969924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2efeb480-e25c-4839-bd64-29faffc3d9da tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:23.970586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2efeb480-e25c-4839-bd64-29faffc3d9da tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:23.971241 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2efeb480-e25c-4839-bd64-29faffc3d9da tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:23.972010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2efeb480-e25c-4839-bd64-29faffc3d9da tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:23.972474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2efeb480-e25c-4839-bd64-29faffc3d9da tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:23.972901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2efeb480-e25c-4839-bd64-29faffc3d9da tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:24.168154 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2efeb480-e25c-4839-bd64-29faffc3d9da tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:24.179566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2efeb480-e25c-4839-bd64-29faffc3d9da tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.223492 Sep 08 14:06:24.180164 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1280/2326] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:23 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:24.184890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ab352dab-c775-4855-8636-40695efce494 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:24.185496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ab352dab-c775-4855-8636-40695efce494 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:24.193143 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab352dab-c775-4855-8636-40695efce494 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:24.194878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab352dab-c775-4855-8636-40695efce494 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:24.195433 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab352dab-c775-4855-8636-40695efce494 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:24.196302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab352dab-c775-4855-8636-40695efce494 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:24.196887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab352dab-c775-4855-8636-40695efce494 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:24.197417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ab352dab-c775-4855-8636-40695efce494 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:24.281888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ab352dab-c775-4855-8636-40695efce494 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:24.292031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ab352dab-c775-4855-8636-40695efce494 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.110532 Sep 08 14:06:24.292857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1281/2327] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:24 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:24.297831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a7cfd54f-1c6e-43dd-b435-8ad3be35fa6d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:24.300104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a7cfd54f-1c6e-43dd-b435-8ad3be35fa6d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:24.307097 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7cfd54f-1c6e-43dd-b435-8ad3be35fa6d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:24.307585 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7cfd54f-1c6e-43dd-b435-8ad3be35fa6d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:24.308063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7cfd54f-1c6e-43dd-b435-8ad3be35fa6d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:24.308762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7cfd54f-1c6e-43dd-b435-8ad3be35fa6d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:24.309217 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7cfd54f-1c6e-43dd-b435-8ad3be35fa6d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:24.309640 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7cfd54f-1c6e-43dd-b435-8ad3be35fa6d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:24.343809 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a7cfd54f-1c6e-43dd-b435-8ad3be35fa6d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049861 Sep 08 14:06:24.344385 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1282/2328] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:24 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:24.349426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0710220e-4767-4b1f-8c1f-c602ae327ebc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:24.352002 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0710220e-4767-4b1f-8c1f-c602ae327ebc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:24.358777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0710220e-4767-4b1f-8c1f-c602ae327ebc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:24.359374 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0710220e-4767-4b1f-8c1f-c602ae327ebc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:24.359928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0710220e-4767-4b1f-8c1f-c602ae327ebc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:24.360697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0710220e-4767-4b1f-8c1f-c602ae327ebc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:24.361197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0710220e-4767-4b1f-8c1f-c602ae327ebc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:24.361673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0710220e-4767-4b1f-8c1f-c602ae327ebc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:24.402180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0710220e-4767-4b1f-8c1f-c602ae327ebc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056493 Sep 08 14:06:24.403240 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1283/2329] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:24 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:25.210656 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ad59fcee-09b5-44b2-91cf-de97359374ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:25.211701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ad59fcee-09b5-44b2-91cf-de97359374ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:25.218367 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad59fcee-09b5-44b2-91cf-de97359374ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:25.218848 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad59fcee-09b5-44b2-91cf-de97359374ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:25.219275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad59fcee-09b5-44b2-91cf-de97359374ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:25.219962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad59fcee-09b5-44b2-91cf-de97359374ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:25.220465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad59fcee-09b5-44b2-91cf-de97359374ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:25.220883 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad59fcee-09b5-44b2-91cf-de97359374ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:25.422268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ad59fcee-09b5-44b2-91cf-de97359374ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:25.431916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ad59fcee-09b5-44b2-91cf-de97359374ca tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.226488 Sep 08 14:06:25.432580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1284/2330] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:25 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:25.437296 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d4b0fe3c-a7a8-4250-9162-9d74c9927626 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:25.437906 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d4b0fe3c-a7a8-4250-9162-9d74c9927626 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:25.444954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4b0fe3c-a7a8-4250-9162-9d74c9927626 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:25.445530 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4b0fe3c-a7a8-4250-9162-9d74c9927626 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:25.446029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4b0fe3c-a7a8-4250-9162-9d74c9927626 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:25.446881 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4b0fe3c-a7a8-4250-9162-9d74c9927626 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:25.447502 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4b0fe3c-a7a8-4250-9162-9d74c9927626 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:25.448042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4b0fe3c-a7a8-4250-9162-9d74c9927626 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:25.543107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d4b0fe3c-a7a8-4250-9162-9d74c9927626 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:25.558464 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d4b0fe3c-a7a8-4250-9162-9d74c9927626 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.124407 Sep 08 14:06:25.559519 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1285/2331] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:25 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:25.569127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b158cd0f-c597-4ee3-9a5c-bf6025f5e318 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:25.573184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b158cd0f-c597-4ee3-9a5c-bf6025f5e318 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:25.584325 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b158cd0f-c597-4ee3-9a5c-bf6025f5e318 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:25.585230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b158cd0f-c597-4ee3-9a5c-bf6025f5e318 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:25.586054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b158cd0f-c597-4ee3-9a5c-bf6025f5e318 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:25.587386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b158cd0f-c597-4ee3-9a5c-bf6025f5e318 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:25.588253 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b158cd0f-c597-4ee3-9a5c-bf6025f5e318 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:25.589062 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b158cd0f-c597-4ee3-9a5c-bf6025f5e318 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:25.630223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b158cd0f-c597-4ee3-9a5c-bf6025f5e318 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069066 Sep 08 14:06:25.631680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1286/2332] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:25 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:25.637781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ae2b2462-c452-4c5e-8ae5-d1febf5e44d3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:25.640923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ae2b2462-c452-4c5e-8ae5-d1febf5e44d3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:25.648097 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae2b2462-c452-4c5e-8ae5-d1febf5e44d3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:25.648701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae2b2462-c452-4c5e-8ae5-d1febf5e44d3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:25.649143 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae2b2462-c452-4c5e-8ae5-d1febf5e44d3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:25.649962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae2b2462-c452-4c5e-8ae5-d1febf5e44d3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:25.650444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae2b2462-c452-4c5e-8ae5-d1febf5e44d3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:25.650866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae2b2462-c452-4c5e-8ae5-d1febf5e44d3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:25.686854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ae2b2462-c452-4c5e-8ae5-d1febf5e44d3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054054 Sep 08 14:06:25.687431 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1287/2333] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:25 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:26.471968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cf36fc13-2ab6-4c36-a247-da677fb6ca5c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:26.473315 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cf36fc13-2ab6-4c36-a247-da677fb6ca5c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:26.486018 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cf36fc13-2ab6-4c36-a247-da677fb6ca5c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:26.486943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cf36fc13-2ab6-4c36-a247-da677fb6ca5c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:26.487738 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cf36fc13-2ab6-4c36-a247-da677fb6ca5c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:26.489165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cf36fc13-2ab6-4c36-a247-da677fb6ca5c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:26.490392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cf36fc13-2ab6-4c36-a247-da677fb6ca5c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:26.491261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cf36fc13-2ab6-4c36-a247-da677fb6ca5c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:26.680742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cf36fc13-2ab6-4c36-a247-da677fb6ca5c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:26.694225 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cf36fc13-2ab6-4c36-a247-da677fb6ca5c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.229326 Sep 08 14:06:26.695388 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1288/2334] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:26 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:26.702838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c98d7cc2-6a6d-47a1-82f9-4f7f75373b86 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:26.703721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c98d7cc2-6a6d-47a1-82f9-4f7f75373b86 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:26.715167 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c98d7cc2-6a6d-47a1-82f9-4f7f75373b86 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:26.715954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c98d7cc2-6a6d-47a1-82f9-4f7f75373b86 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:26.716530 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c98d7cc2-6a6d-47a1-82f9-4f7f75373b86 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:26.717777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c98d7cc2-6a6d-47a1-82f9-4f7f75373b86 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:26.718515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c98d7cc2-6a6d-47a1-82f9-4f7f75373b86 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:26.719280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c98d7cc2-6a6d-47a1-82f9-4f7f75373b86 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:26.815657 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c98d7cc2-6a6d-47a1-82f9-4f7f75373b86 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:26.824757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c98d7cc2-6a6d-47a1-82f9-4f7f75373b86 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.127991 Sep 08 14:06:26.825343 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1289/2335] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:26 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:26.830432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-78af794a-742a-484b-bc67-1d6707f2078d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:26.832634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-78af794a-742a-484b-bc67-1d6707f2078d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:26.834586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:26.840347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-78af794a-742a-484b-bc67-1d6707f2078d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:26.840850 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-78af794a-742a-484b-bc67-1d6707f2078d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:26.841295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-78af794a-742a-484b-bc67-1d6707f2078d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:26.842068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-78af794a-742a-484b-bc67-1d6707f2078d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:26.842812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-78af794a-742a-484b-bc67-1d6707f2078d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:26.843255 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-78af794a-742a-484b-bc67-1d6707f2078d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:26.882988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-78af794a-742a-484b-bc67-1d6707f2078d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056548 Sep 08 14:06:26.883698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1290/2336] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:26 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:26.889175 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-26efc40f-1e7d-4f44-a658-ac96c182e9fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:26.891566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-26efc40f-1e7d-4f44-a658-ac96c182e9fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:26.901027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26efc40f-1e7d-4f44-a658-ac96c182e9fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:26.901822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26efc40f-1e7d-4f44-a658-ac96c182e9fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:26.902599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26efc40f-1e7d-4f44-a658-ac96c182e9fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:26.903744 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26efc40f-1e7d-4f44-a658-ac96c182e9fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:26.904319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26efc40f-1e7d-4f44-a658-ac96c182e9fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:26.904765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-26efc40f-1e7d-4f44-a658-ac96c182e9fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:26.943942 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-26efc40f-1e7d-4f44-a658-ac96c182e9fc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058332 Sep 08 14:06:26.944604 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1291/2337] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:26 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:27.742578 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d525b80f-f445-4908-b781-b3b3eeefca1d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:27.743370 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d525b80f-f445-4908-b781-b3b3eeefca1d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:27.752333 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d525b80f-f445-4908-b781-b3b3eeefca1d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:27.753239 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d525b80f-f445-4908-b781-b3b3eeefca1d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:27.753946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d525b80f-f445-4908-b781-b3b3eeefca1d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:27.755257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d525b80f-f445-4908-b781-b3b3eeefca1d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:27.756787 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d525b80f-f445-4908-b781-b3b3eeefca1d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:27.757552 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d525b80f-f445-4908-b781-b3b3eeefca1d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:27.930654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d525b80f-f445-4908-b781-b3b3eeefca1d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:27.939851 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d525b80f-f445-4908-b781-b3b3eeefca1d tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.202572 Sep 08 14:06:27.940436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1292/2338] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:27 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:27.945805 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5db7f44c-38c9-4c0b-aaac-ddd17df18a37 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:27.946654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5db7f44c-38c9-4c0b-aaac-ddd17df18a37 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:27.955461 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5db7f44c-38c9-4c0b-aaac-ddd17df18a37 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:27.956384 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5db7f44c-38c9-4c0b-aaac-ddd17df18a37 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:27.957026 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5db7f44c-38c9-4c0b-aaac-ddd17df18a37 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:27.957895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5db7f44c-38c9-4c0b-aaac-ddd17df18a37 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:27.958566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5db7f44c-38c9-4c0b-aaac-ddd17df18a37 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:27.959275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5db7f44c-38c9-4c0b-aaac-ddd17df18a37 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:27.965370 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:28.037390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5db7f44c-38c9-4c0b-aaac-ddd17df18a37 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:28.050850 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5db7f44c-38c9-4c0b-aaac-ddd17df18a37 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.109316 Sep 08 14:06:28.051455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1293/2339] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:27 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:28.057583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2cb8070f-967b-4b27-a965-7991553787be tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:28.059849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2cb8070f-967b-4b27-a965-7991553787be tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:28.066621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2cb8070f-967b-4b27-a965-7991553787be tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:28.067129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2cb8070f-967b-4b27-a965-7991553787be tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:28.069298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2cb8070f-967b-4b27-a965-7991553787be tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:28.069298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2cb8070f-967b-4b27-a965-7991553787be tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:28.069634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2cb8070f-967b-4b27-a965-7991553787be tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:28.070322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2cb8070f-967b-4b27-a965-7991553787be tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:28.122401 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2cb8070f-967b-4b27-a965-7991553787be tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067480 Sep 08 14:06:28.123634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1294/2340] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:28 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:28.129840 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-dadbd40c-13a8-42bf-b7f9-9b0c8e7d5e0e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:28.134011 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-dadbd40c-13a8-42bf-b7f9-9b0c8e7d5e0e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:28.141725 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dadbd40c-13a8-42bf-b7f9-9b0c8e7d5e0e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:28.142303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dadbd40c-13a8-42bf-b7f9-9b0c8e7d5e0e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:28.142815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dadbd40c-13a8-42bf-b7f9-9b0c8e7d5e0e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:28.143659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dadbd40c-13a8-42bf-b7f9-9b0c8e7d5e0e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:28.144287 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dadbd40c-13a8-42bf-b7f9-9b0c8e7d5e0e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:28.144804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dadbd40c-13a8-42bf-b7f9-9b0c8e7d5e0e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:28.186427 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-dadbd40c-13a8-42bf-b7f9-9b0c8e7d5e0e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061588 Sep 08 14:06:28.187566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1295/2341] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:28 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:28.970623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ad58ec8a-8d7e-48bd-b6b5-5ce24f807de7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:28.971613 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ad58ec8a-8d7e-48bd-b6b5-5ce24f807de7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:28.983614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad58ec8a-8d7e-48bd-b6b5-5ce24f807de7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:28.984430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad58ec8a-8d7e-48bd-b6b5-5ce24f807de7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:28.985229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad58ec8a-8d7e-48bd-b6b5-5ce24f807de7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:28.986518 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad58ec8a-8d7e-48bd-b6b5-5ce24f807de7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:28.987348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad58ec8a-8d7e-48bd-b6b5-5ce24f807de7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:28.988115 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad58ec8a-8d7e-48bd-b6b5-5ce24f807de7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:29.182926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ad58ec8a-8d7e-48bd-b6b5-5ce24f807de7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:29.198840 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ad58ec8a-8d7e-48bd-b6b5-5ce24f807de7 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.232705 Sep 08 14:06:29.199191 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1296/2342] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:28 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:29.208448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fa7d1c57-ba92-4a43-bac4-e934f1549c38 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:29.210388 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fa7d1c57-ba92-4a43-bac4-e934f1549c38 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:29.220572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fa7d1c57-ba92-4a43-bac4-e934f1549c38 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:29.221224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fa7d1c57-ba92-4a43-bac4-e934f1549c38 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:29.221864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fa7d1c57-ba92-4a43-bac4-e934f1549c38 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:29.223274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fa7d1c57-ba92-4a43-bac4-e934f1549c38 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:29.223935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fa7d1c57-ba92-4a43-bac4-e934f1549c38 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:29.224784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fa7d1c57-ba92-4a43-bac4-e934f1549c38 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:29.312508 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fa7d1c57-ba92-4a43-bac4-e934f1549c38 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:29.323015 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fa7d1c57-ba92-4a43-bac4-e934f1549c38 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.122360 Sep 08 14:06:29.323734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1297/2343] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:29 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:29.329143 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bb9d4417-8583-47f5-90c2-d1a1e3ee487f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:29.331979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bb9d4417-8583-47f5-90c2-d1a1e3ee487f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:29.340129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb9d4417-8583-47f5-90c2-d1a1e3ee487f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:29.340818 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb9d4417-8583-47f5-90c2-d1a1e3ee487f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:29.341515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb9d4417-8583-47f5-90c2-d1a1e3ee487f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:29.342696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb9d4417-8583-47f5-90c2-d1a1e3ee487f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:29.343549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb9d4417-8583-47f5-90c2-d1a1e3ee487f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:29.344145 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bb9d4417-8583-47f5-90c2-d1a1e3ee487f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:29.383462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bb9d4417-8583-47f5-90c2-d1a1e3ee487f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058472 Sep 08 14:06:29.384056 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1298/2344] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:29 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:29.388896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cb38da45-a89c-4b7f-848c-1b1cd9be7e17 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:29.392487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cb38da45-a89c-4b7f-848c-1b1cd9be7e17 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:29.402336 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cb38da45-a89c-4b7f-848c-1b1cd9be7e17 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:29.403108 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cb38da45-a89c-4b7f-848c-1b1cd9be7e17 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:29.403943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cb38da45-a89c-4b7f-848c-1b1cd9be7e17 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:29.404793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cb38da45-a89c-4b7f-848c-1b1cd9be7e17 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:29.405326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cb38da45-a89c-4b7f-848c-1b1cd9be7e17 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:29.405757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cb38da45-a89c-4b7f-848c-1b1cd9be7e17 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:29.445559 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cb38da45-a89c-4b7f-848c-1b1cd9be7e17 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060436 Sep 08 14:06:29.446158 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1299/2345] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:29 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:30.250178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2621fa5b-cba2-4f17-97a4-270a3ca204d5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:30.251510 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2621fa5b-cba2-4f17-97a4-270a3ca204d5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:30.262814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2621fa5b-cba2-4f17-97a4-270a3ca204d5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:30.263796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2621fa5b-cba2-4f17-97a4-270a3ca204d5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:30.264745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2621fa5b-cba2-4f17-97a4-270a3ca204d5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:30.266039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2621fa5b-cba2-4f17-97a4-270a3ca204d5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:30.266866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2621fa5b-cba2-4f17-97a4-270a3ca204d5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:30.268038 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2621fa5b-cba2-4f17-97a4-270a3ca204d5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:30.483407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2621fa5b-cba2-4f17-97a4-270a3ca204d5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:30.498694 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2621fa5b-cba2-4f17-97a4-270a3ca204d5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.255647 Sep 08 14:06:30.505713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1300/2346] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:30 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:30.506184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ad9dd8f4-799e-4032-bb24-d2fdbd8c0373 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:30.506807 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ad9dd8f4-799e-4032-bb24-d2fdbd8c0373 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:30.515846 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad9dd8f4-799e-4032-bb24-d2fdbd8c0373 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:30.516528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad9dd8f4-799e-4032-bb24-d2fdbd8c0373 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:30.517326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad9dd8f4-799e-4032-bb24-d2fdbd8c0373 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:30.518294 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad9dd8f4-799e-4032-bb24-d2fdbd8c0373 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:30.518921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad9dd8f4-799e-4032-bb24-d2fdbd8c0373 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:30.519742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad9dd8f4-799e-4032-bb24-d2fdbd8c0373 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:30.609560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ad9dd8f4-799e-4032-bb24-d2fdbd8c0373 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:30.623659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ad9dd8f4-799e-4032-bb24-d2fdbd8c0373 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.122089 Sep 08 14:06:30.625626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1301/2347] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:30 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:30.633432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2c769856-824d-4d8e-97fd-553e5f7a5cda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:30.637069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2c769856-824d-4d8e-97fd-553e5f7a5cda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:30.645184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c769856-824d-4d8e-97fd-553e5f7a5cda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:30.645713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c769856-824d-4d8e-97fd-553e5f7a5cda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:30.646776 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c769856-824d-4d8e-97fd-553e5f7a5cda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:30.647546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c769856-824d-4d8e-97fd-553e5f7a5cda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:30.648040 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c769856-824d-4d8e-97fd-553e5f7a5cda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:30.648485 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c769856-824d-4d8e-97fd-553e5f7a5cda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:30.682874 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2c769856-824d-4d8e-97fd-553e5f7a5cda tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055909 Sep 08 14:06:30.683460 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1302/2348] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:30 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:30.689846 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8c267d12-d5d5-4511-9298-ae448b9b214a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:30.692121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8c267d12-d5d5-4511-9298-ae448b9b214a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:30.699031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c267d12-d5d5-4511-9298-ae448b9b214a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:30.699529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c267d12-d5d5-4511-9298-ae448b9b214a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:30.699968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c267d12-d5d5-4511-9298-ae448b9b214a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:30.700681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c267d12-d5d5-4511-9298-ae448b9b214a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:30.701172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c267d12-d5d5-4511-9298-ae448b9b214a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:30.701688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8c267d12-d5d5-4511-9298-ae448b9b214a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:30.736371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8c267d12-d5d5-4511-9298-ae448b9b214a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051810 Sep 08 14:06:30.736955 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1303/2349] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:30 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:31.532236 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b5039b46-183b-4e11-a5a5-1155fe8d943e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:31.533561 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b5039b46-183b-4e11-a5a5-1155fe8d943e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:31.543514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5039b46-183b-4e11-a5a5-1155fe8d943e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:31.544015 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5039b46-183b-4e11-a5a5-1155fe8d943e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:31.544694 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5039b46-183b-4e11-a5a5-1155fe8d943e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:31.545733 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5039b46-183b-4e11-a5a5-1155fe8d943e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:31.546033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5039b46-183b-4e11-a5a5-1155fe8d943e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:31.546484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b5039b46-183b-4e11-a5a5-1155fe8d943e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:31.743110 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b5039b46-183b-4e11-a5a5-1155fe8d943e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:31.755514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b5039b46-183b-4e11-a5a5-1155fe8d943e tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.227666 Sep 08 14:06:31.756527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1304/2350] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:31 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:31.762411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5b1030fc-0237-4307-aa98-34e92c193d16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:31.763023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5b1030fc-0237-4307-aa98-34e92c193d16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:31.770955 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b1030fc-0237-4307-aa98-34e92c193d16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:31.772462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b1030fc-0237-4307-aa98-34e92c193d16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:31.773213 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b1030fc-0237-4307-aa98-34e92c193d16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:31.774341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b1030fc-0237-4307-aa98-34e92c193d16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:31.775155 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b1030fc-0237-4307-aa98-34e92c193d16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:31.780043 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b1030fc-0237-4307-aa98-34e92c193d16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:31.856518 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5b1030fc-0237-4307-aa98-34e92c193d16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:31.867677 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5b1030fc-0237-4307-aa98-34e92c193d16 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.109986 Sep 08 14:06:31.868303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1305/2351] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:31 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:31.873267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e5f57dfe-1704-44b8-b5f4-113964194174 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:31.875887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e5f57dfe-1704-44b8-b5f4-113964194174 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:31.883672 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e5f57dfe-1704-44b8-b5f4-113964194174 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:31.884191 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e5f57dfe-1704-44b8-b5f4-113964194174 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:31.884865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e5f57dfe-1704-44b8-b5f4-113964194174 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:31.885825 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e5f57dfe-1704-44b8-b5f4-113964194174 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:31.886324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e5f57dfe-1704-44b8-b5f4-113964194174 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:31.886758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e5f57dfe-1704-44b8-b5f4-113964194174 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:31.924412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e5f57dfe-1704-44b8-b5f4-113964194174 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054960 Sep 08 14:06:31.933009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1306/2352] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:31 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:31.933009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-70410266-5e76-4685-942a-3182694fc3a6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:31.933502 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-70410266-5e76-4685-942a-3182694fc3a6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:31.939703 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70410266-5e76-4685-942a-3182694fc3a6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:31.940351 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70410266-5e76-4685-942a-3182694fc3a6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:31.940916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70410266-5e76-4685-942a-3182694fc3a6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:31.941734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70410266-5e76-4685-942a-3182694fc3a6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:31.942359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70410266-5e76-4685-942a-3182694fc3a6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:31.943123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-70410266-5e76-4685-942a-3182694fc3a6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:31.983122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-70410266-5e76-4685-942a-3182694fc3a6 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056194 Sep 08 14:06:31.983486 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1307/2353] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:31 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:32.788587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-353b7980-74ea-4695-b244-93d0b5730b27 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:32.789709 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-353b7980-74ea-4695-b244-93d0b5730b27 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:32.802228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-353b7980-74ea-4695-b244-93d0b5730b27 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:32.803593 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-353b7980-74ea-4695-b244-93d0b5730b27 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:32.804480 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-353b7980-74ea-4695-b244-93d0b5730b27 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:32.805714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-353b7980-74ea-4695-b244-93d0b5730b27 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:32.806566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-353b7980-74ea-4695-b244-93d0b5730b27 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:32.807336 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-353b7980-74ea-4695-b244-93d0b5730b27 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:32.976367 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-353b7980-74ea-4695-b244-93d0b5730b27 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:32.985642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-353b7980-74ea-4695-b244-93d0b5730b27 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.204961 Sep 08 14:06:32.992936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1308/2354] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:32 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:33.083426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9628ee2b-9db9-4bf5-aa3d-ea8ce48bc6da tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:33.084639 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9628ee2b-9db9-4bf5-aa3d-ea8ce48bc6da tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:33.093894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9628ee2b-9db9-4bf5-aa3d-ea8ce48bc6da tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:33.094489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9628ee2b-9db9-4bf5-aa3d-ea8ce48bc6da tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:33.095173 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9628ee2b-9db9-4bf5-aa3d-ea8ce48bc6da tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:33.095904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9628ee2b-9db9-4bf5-aa3d-ea8ce48bc6da tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:33.096384 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9628ee2b-9db9-4bf5-aa3d-ea8ce48bc6da tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:33.096817 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9628ee2b-9db9-4bf5-aa3d-ea8ce48bc6da tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:33.189640 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9628ee2b-9db9-4bf5-aa3d-ea8ce48bc6da tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:33.192686 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:33.202974 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9628ee2b-9db9-4bf5-aa3d-ea8ce48bc6da tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.215442 Sep 08 14:06:33.204747 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1309/2355] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:32 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:33.210019 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-76f86770-a236-4506-b212-959faa9cacc1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:33.212656 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-76f86770-a236-4506-b212-959faa9cacc1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:33.221081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-76f86770-a236-4506-b212-959faa9cacc1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:33.221667 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-76f86770-a236-4506-b212-959faa9cacc1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:33.222313 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-76f86770-a236-4506-b212-959faa9cacc1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:33.223437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-76f86770-a236-4506-b212-959faa9cacc1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:33.224266 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-76f86770-a236-4506-b212-959faa9cacc1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:33.225007 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-76f86770-a236-4506-b212-959faa9cacc1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:33.280074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-76f86770-a236-4506-b212-959faa9cacc1 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.075252 Sep 08 14:06:33.280679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1310/2356] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:33 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:33.288868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f77b65ea-2aeb-402b-89a5-c0e551d097c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:33.291959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f77b65ea-2aeb-402b-89a5-c0e551d097c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:33.307804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f77b65ea-2aeb-402b-89a5-c0e551d097c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:33.308494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f77b65ea-2aeb-402b-89a5-c0e551d097c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:33.309107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f77b65ea-2aeb-402b-89a5-c0e551d097c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:33.310378 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f77b65ea-2aeb-402b-89a5-c0e551d097c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:33.311332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f77b65ea-2aeb-402b-89a5-c0e551d097c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:33.312101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f77b65ea-2aeb-402b-89a5-c0e551d097c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:33.368531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f77b65ea-2aeb-402b-89a5-c0e551d097c2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.086375 Sep 08 14:06:33.369471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1311/2357] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:33 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:34.017656 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f41709ad-491e-444b-b399-9d5dd39e58b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:34.018796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f41709ad-491e-444b-b399-9d5dd39e58b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:34.028637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f41709ad-491e-444b-b399-9d5dd39e58b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:34.029133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f41709ad-491e-444b-b399-9d5dd39e58b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:34.029570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f41709ad-491e-444b-b399-9d5dd39e58b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:34.030670 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f41709ad-491e-444b-b399-9d5dd39e58b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:34.031362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f41709ad-491e-444b-b399-9d5dd39e58b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:34.031552 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f41709ad-491e-444b-b399-9d5dd39e58b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:34.234722 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f41709ad-491e-444b-b399-9d5dd39e58b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:34.244841 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f41709ad-491e-444b-b399-9d5dd39e58b8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.231439 Sep 08 14:06:34.245615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1312/2358] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:34 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:34.250442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-31424bdc-7810-4dc2-96cf-f1ad2954dd7d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:34.251053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-31424bdc-7810-4dc2-96cf-f1ad2954dd7d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:34.259504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31424bdc-7810-4dc2-96cf-f1ad2954dd7d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:34.259994 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31424bdc-7810-4dc2-96cf-f1ad2954dd7d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:34.260443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31424bdc-7810-4dc2-96cf-f1ad2954dd7d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:34.261137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31424bdc-7810-4dc2-96cf-f1ad2954dd7d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:34.261587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31424bdc-7810-4dc2-96cf-f1ad2954dd7d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:34.262017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-31424bdc-7810-4dc2-96cf-f1ad2954dd7d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:34.341324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-31424bdc-7810-4dc2-96cf-f1ad2954dd7d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:34.352558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-31424bdc-7810-4dc2-96cf-f1ad2954dd7d tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.105729 Sep 08 14:06:34.353142 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1313/2359] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:34 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:34.358033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-dbfdf759-9bef-4f50-8695-3738fbbd885b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:34.360604 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-dbfdf759-9bef-4f50-8695-3738fbbd885b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:34.370127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dbfdf759-9bef-4f50-8695-3738fbbd885b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:34.370969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dbfdf759-9bef-4f50-8695-3738fbbd885b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:34.371702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dbfdf759-9bef-4f50-8695-3738fbbd885b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:34.372943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dbfdf759-9bef-4f50-8695-3738fbbd885b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:34.373699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dbfdf759-9bef-4f50-8695-3738fbbd885b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:34.374471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dbfdf759-9bef-4f50-8695-3738fbbd885b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:34.412236 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-dbfdf759-9bef-4f50-8695-3738fbbd885b tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057342 Sep 08 14:06:34.412845 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1314/2360] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:34 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:34.417581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-63b8eb32-09c8-4902-8eef-87fa3183096d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:34.419838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-63b8eb32-09c8-4902-8eef-87fa3183096d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:34.426353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-63b8eb32-09c8-4902-8eef-87fa3183096d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:34.426839 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-63b8eb32-09c8-4902-8eef-87fa3183096d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:34.428534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-63b8eb32-09c8-4902-8eef-87fa3183096d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:34.429245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-63b8eb32-09c8-4902-8eef-87fa3183096d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:34.429701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-63b8eb32-09c8-4902-8eef-87fa3183096d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:34.430126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-63b8eb32-09c8-4902-8eef-87fa3183096d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:34.463885 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-63b8eb32-09c8-4902-8eef-87fa3183096d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049938 Sep 08 14:06:34.464424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1315/2361] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:34 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:35.283931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3fce7469-2490-470a-9078-6a749e094729 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:35.284821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3fce7469-2490-470a-9078-6a749e094729 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:35.292953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3fce7469-2490-470a-9078-6a749e094729 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:35.293451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3fce7469-2490-470a-9078-6a749e094729 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:35.293890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3fce7469-2490-470a-9078-6a749e094729 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:35.294693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3fce7469-2490-470a-9078-6a749e094729 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:35.295154 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3fce7469-2490-470a-9078-6a749e094729 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:35.295582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3fce7469-2490-470a-9078-6a749e094729 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:35.507635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3fce7469-2490-470a-9078-6a749e094729 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:35.516027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3fce7469-2490-470a-9078-6a749e094729 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.236213 Sep 08 14:06:35.516854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1316/2362] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:35 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:35.522661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3c1a6c10-3267-40c4-a274-da3e126b93eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:35.523420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3c1a6c10-3267-40c4-a274-da3e126b93eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:35.538202 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3c1a6c10-3267-40c4-a274-da3e126b93eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:35.538905 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3c1a6c10-3267-40c4-a274-da3e126b93eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:35.539575 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3c1a6c10-3267-40c4-a274-da3e126b93eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:35.540689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3c1a6c10-3267-40c4-a274-da3e126b93eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:35.541429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3c1a6c10-3267-40c4-a274-da3e126b93eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:35.542090 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3c1a6c10-3267-40c4-a274-da3e126b93eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:35.634962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3c1a6c10-3267-40c4-a274-da3e126b93eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:35.648634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3c1a6c10-3267-40c4-a274-da3e126b93eb tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.130613 Sep 08 14:06:35.649396 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1317/2363] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:35 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:35.657422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a7b1adb8-e05b-4c0d-8a2e-4c258a3240ab tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:35.660937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a7b1adb8-e05b-4c0d-8a2e-4c258a3240ab tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:35.671790 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7b1adb8-e05b-4c0d-8a2e-4c258a3240ab tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:35.672632 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7b1adb8-e05b-4c0d-8a2e-4c258a3240ab tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:35.673081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7b1adb8-e05b-4c0d-8a2e-4c258a3240ab tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:35.673979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7b1adb8-e05b-4c0d-8a2e-4c258a3240ab tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:35.674646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7b1adb8-e05b-4c0d-8a2e-4c258a3240ab tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:35.675092 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a7b1adb8-e05b-4c0d-8a2e-4c258a3240ab tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:35.716119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a7b1adb8-e05b-4c0d-8a2e-4c258a3240ab tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.065428 Sep 08 14:06:35.717047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1318/2364] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:35 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:35.722245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-59bd9962-f68e-4fdf-9c26-32228d05cacb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:35.724527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-59bd9962-f68e-4fdf-9c26-32228d05cacb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:35.730940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-59bd9962-f68e-4fdf-9c26-32228d05cacb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:35.731425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-59bd9962-f68e-4fdf-9c26-32228d05cacb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:35.731856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-59bd9962-f68e-4fdf-9c26-32228d05cacb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:35.732549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-59bd9962-f68e-4fdf-9c26-32228d05cacb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:35.733151 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-59bd9962-f68e-4fdf-9c26-32228d05cacb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:35.733571 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-59bd9962-f68e-4fdf-9c26-32228d05cacb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:35.768143 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-59bd9962-f68e-4fdf-9c26-32228d05cacb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049798 Sep 08 14:06:35.769006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1319/2365] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:35 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:36.555860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9792830a-8709-4547-afe5-3d78fa93d5d8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:36.556930 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9792830a-8709-4547-afe5-3d78fa93d5d8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:36.566472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9792830a-8709-4547-afe5-3d78fa93d5d8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:36.567206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9792830a-8709-4547-afe5-3d78fa93d5d8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:36.570659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9792830a-8709-4547-afe5-3d78fa93d5d8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:36.570659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9792830a-8709-4547-afe5-3d78fa93d5d8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:36.570659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9792830a-8709-4547-afe5-3d78fa93d5d8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:36.571222 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9792830a-8709-4547-afe5-3d78fa93d5d8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:36.774877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9792830a-8709-4547-afe5-3d78fa93d5d8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:36.790663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9792830a-8709-4547-afe5-3d78fa93d5d8 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.247427 Sep 08 14:06:36.791951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1320/2366] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:36 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:36.801143 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b740e00e-fc90-46a1-9e38-3be60cd11883 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:36.802840 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b740e00e-fc90-46a1-9e38-3be60cd11883 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:36.814313 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b740e00e-fc90-46a1-9e38-3be60cd11883 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:36.815109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b740e00e-fc90-46a1-9e38-3be60cd11883 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:36.815896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b740e00e-fc90-46a1-9e38-3be60cd11883 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:36.822953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b740e00e-fc90-46a1-9e38-3be60cd11883 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:36.822953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b740e00e-fc90-46a1-9e38-3be60cd11883 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:36.822953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b740e00e-fc90-46a1-9e38-3be60cd11883 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:36.920997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b740e00e-fc90-46a1-9e38-3be60cd11883 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:36.931100 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b740e00e-fc90-46a1-9e38-3be60cd11883 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.137424 Sep 08 14:06:36.931880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1321/2367] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:36 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:36.938070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3bddc361-969a-4346-80df-6abae79815f2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:36.941416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3bddc361-969a-4346-80df-6abae79815f2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:36.952091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3bddc361-969a-4346-80df-6abae79815f2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:36.953005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3bddc361-969a-4346-80df-6abae79815f2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:36.953698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3bddc361-969a-4346-80df-6abae79815f2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:36.954462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3bddc361-969a-4346-80df-6abae79815f2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:36.954926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3bddc361-969a-4346-80df-6abae79815f2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:36.955391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3bddc361-969a-4346-80df-6abae79815f2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:36.994264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3bddc361-969a-4346-80df-6abae79815f2 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060998 Sep 08 14:06:36.994923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1322/2368] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:36 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:37.000256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3624dc17-7da9-4ce8-a6d9-fd4da4b565bb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:37.002643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3624dc17-7da9-4ce8-a6d9-fd4da4b565bb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:37.009546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3624dc17-7da9-4ce8-a6d9-fd4da4b565bb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:37.010034 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3624dc17-7da9-4ce8-a6d9-fd4da4b565bb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:37.010501 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3624dc17-7da9-4ce8-a6d9-fd4da4b565bb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:37.011197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3624dc17-7da9-4ce8-a6d9-fd4da4b565bb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:37.011686 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3624dc17-7da9-4ce8-a6d9-fd4da4b565bb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:37.012167 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3624dc17-7da9-4ce8-a6d9-fd4da4b565bb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:37.050547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3624dc17-7da9-4ce8-a6d9-fd4da4b565bb tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054346 Sep 08 14:06:37.051248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1323/2369] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:36 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:37.822453 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cbc6dcc3-60aa-46af-a96e-098c2b96214b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:37.823450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cbc6dcc3-60aa-46af-a96e-098c2b96214b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:37.834366 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbc6dcc3-60aa-46af-a96e-098c2b96214b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:37.835181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbc6dcc3-60aa-46af-a96e-098c2b96214b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:37.835784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbc6dcc3-60aa-46af-a96e-098c2b96214b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:37.840424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbc6dcc3-60aa-46af-a96e-098c2b96214b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:37.841015 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbc6dcc3-60aa-46af-a96e-098c2b96214b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:37.841643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbc6dcc3-60aa-46af-a96e-098c2b96214b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:38.009083 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cbc6dcc3-60aa-46af-a96e-098c2b96214b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:38.019029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cbc6dcc3-60aa-46af-a96e-098c2b96214b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.203907 Sep 08 14:06:38.019861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1324/2370] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:37 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:38.034377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fab2ffb4-df45-451a-9f36-9566b16050e1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:38.034377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fab2ffb4-df45-451a-9f36-9566b16050e1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:38.034377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fab2ffb4-df45-451a-9f36-9566b16050e1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:38.034377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fab2ffb4-df45-451a-9f36-9566b16050e1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:38.037074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fab2ffb4-df45-451a-9f36-9566b16050e1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:38.037074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fab2ffb4-df45-451a-9f36-9566b16050e1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:38.037074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fab2ffb4-df45-451a-9f36-9566b16050e1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:38.037074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fab2ffb4-df45-451a-9f36-9566b16050e1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:38.107066 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fab2ffb4-df45-451a-9f36-9566b16050e1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:38.117934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fab2ffb4-df45-451a-9f36-9566b16050e1 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.096286 Sep 08 14:06:38.118929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1325/2371] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:38 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:38.124035 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d484265a-1397-4a6e-9538-862a7fb6eaae tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:38.126446 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d484265a-1397-4a6e-9538-862a7fb6eaae tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:38.133958 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d484265a-1397-4a6e-9538-862a7fb6eaae tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:38.134762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d484265a-1397-4a6e-9538-862a7fb6eaae tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:38.135341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d484265a-1397-4a6e-9538-862a7fb6eaae tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:38.136151 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d484265a-1397-4a6e-9538-862a7fb6eaae tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:38.136674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d484265a-1397-4a6e-9538-862a7fb6eaae tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:38.137188 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d484265a-1397-4a6e-9538-862a7fb6eaae tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:38.155171 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:38.177949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d484265a-1397-4a6e-9538-862a7fb6eaae tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057864 Sep 08 14:06:38.178720 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1326/2372] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:38 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:38.184400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bf5c54a2-a51f-413f-804c-0173360b6743 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:38.186800 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bf5c54a2-a51f-413f-804c-0173360b6743 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:38.193529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bf5c54a2-a51f-413f-804c-0173360b6743 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:38.194020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bf5c54a2-a51f-413f-804c-0173360b6743 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:38.194753 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bf5c54a2-a51f-413f-804c-0173360b6743 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:38.195618 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bf5c54a2-a51f-413f-804c-0173360b6743 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:38.196083 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bf5c54a2-a51f-413f-804c-0173360b6743 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:38.196512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bf5c54a2-a51f-413f-804c-0173360b6743 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:38.241361 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bf5c54a2-a51f-413f-804c-0173360b6743 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.061405 Sep 08 14:06:38.241935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1327/2373] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:38 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:39.055537 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-06aa558f-9e8d-47fc-9296-deb788511ffc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:39.056239 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-06aa558f-9e8d-47fc-9296-deb788511ffc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:39.063734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06aa558f-9e8d-47fc-9296-deb788511ffc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:39.064500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06aa558f-9e8d-47fc-9296-deb788511ffc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:39.065178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06aa558f-9e8d-47fc-9296-deb788511ffc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:39.066478 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06aa558f-9e8d-47fc-9296-deb788511ffc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:39.067051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06aa558f-9e8d-47fc-9296-deb788511ffc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:39.068115 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-06aa558f-9e8d-47fc-9296-deb788511ffc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:39.249766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-06aa558f-9e8d-47fc-9296-deb788511ffc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:39.263566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-06aa558f-9e8d-47fc-9296-deb788511ffc tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.212040 Sep 08 14:06:39.264559 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1328/2374] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:39 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:39.272237 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-73c8dc85-592d-4660-a893-9d4d6006661f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:39.273248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-73c8dc85-592d-4660-a893-9d4d6006661f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:39.281567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73c8dc85-592d-4660-a893-9d4d6006661f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:39.282168 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73c8dc85-592d-4660-a893-9d4d6006661f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:39.282642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73c8dc85-592d-4660-a893-9d4d6006661f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:39.283375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73c8dc85-592d-4660-a893-9d4d6006661f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:39.283832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73c8dc85-592d-4660-a893-9d4d6006661f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:39.284354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-73c8dc85-592d-4660-a893-9d4d6006661f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:39.361089 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-73c8dc85-592d-4660-a893-9d4d6006661f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:39.374491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-73c8dc85-592d-4660-a893-9d4d6006661f tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.108192 Sep 08 14:06:39.375368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1329/2375] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:39 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:39.384088 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-64b01437-b874-4a0f-89f6-b6615f98ab1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:39.387373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-64b01437-b874-4a0f-89f6-b6615f98ab1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:39.395557 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-64b01437-b874-4a0f-89f6-b6615f98ab1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:39.396215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-64b01437-b874-4a0f-89f6-b6615f98ab1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:39.396655 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-64b01437-b874-4a0f-89f6-b6615f98ab1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:39.397357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-64b01437-b874-4a0f-89f6-b6615f98ab1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:39.397862 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-64b01437-b874-4a0f-89f6-b6615f98ab1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:39.398384 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-64b01437-b874-4a0f-89f6-b6615f98ab1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:39.450524 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-64b01437-b874-4a0f-89f6-b6615f98ab1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.073359 Sep 08 14:06:39.451493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1330/2376] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:39 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:39.460137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b7b063d7-e942-454f-88df-1cc436387603 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:39.464100 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b7b063d7-e942-454f-88df-1cc436387603 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:39.474078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7b063d7-e942-454f-88df-1cc436387603 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:39.474927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7b063d7-e942-454f-88df-1cc436387603 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:39.476209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7b063d7-e942-454f-88df-1cc436387603 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:39.476967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7b063d7-e942-454f-88df-1cc436387603 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:39.477420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7b063d7-e942-454f-88df-1cc436387603 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:39.477895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7b063d7-e942-454f-88df-1cc436387603 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:39.518528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b7b063d7-e942-454f-88df-1cc436387603 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.065283 Sep 08 14:06:39.519118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1331/2377] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:39 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:40.300678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ac0268c1-e448-4a5b-8fcb-addda3005484 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:40.302220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ac0268c1-e448-4a5b-8fcb-addda3005484 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:40.309419 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ac0268c1-e448-4a5b-8fcb-addda3005484 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:40.309962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ac0268c1-e448-4a5b-8fcb-addda3005484 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:40.310496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ac0268c1-e448-4a5b-8fcb-addda3005484 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:40.311244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ac0268c1-e448-4a5b-8fcb-addda3005484 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:40.311869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ac0268c1-e448-4a5b-8fcb-addda3005484 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:40.312341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ac0268c1-e448-4a5b-8fcb-addda3005484 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:40.512265 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ac0268c1-e448-4a5b-8fcb-addda3005484 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:40.524191 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ac0268c1-e448-4a5b-8fcb-addda3005484 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.227222 Sep 08 14:06:40.525607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1332/2378] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:40 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:40.532580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-87bd87e6-eae2-4737-8f63-fee7a1f43740 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:40.533290 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-87bd87e6-eae2-4737-8f63-fee7a1f43740 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:40.541587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-87bd87e6-eae2-4737-8f63-fee7a1f43740 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:40.542190 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-87bd87e6-eae2-4737-8f63-fee7a1f43740 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:40.542716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-87bd87e6-eae2-4737-8f63-fee7a1f43740 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:40.543622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-87bd87e6-eae2-4737-8f63-fee7a1f43740 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:40.544153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-87bd87e6-eae2-4737-8f63-fee7a1f43740 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:40.544651 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-87bd87e6-eae2-4737-8f63-fee7a1f43740 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:40.635166 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-87bd87e6-eae2-4737-8f63-fee7a1f43740 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:40.644119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-87bd87e6-eae2-4737-8f63-fee7a1f43740 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.115631 Sep 08 14:06:40.644792 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1333/2379] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:40 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:40.653548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-23d90ee3-864c-4d97-b3b5-cc29a797103a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:40.657304 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-23d90ee3-864c-4d97-b3b5-cc29a797103a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:40.668678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23d90ee3-864c-4d97-b3b5-cc29a797103a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:40.669567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23d90ee3-864c-4d97-b3b5-cc29a797103a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:40.670398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23d90ee3-864c-4d97-b3b5-cc29a797103a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:40.671663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23d90ee3-864c-4d97-b3b5-cc29a797103a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:40.672514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23d90ee3-864c-4d97-b3b5-cc29a797103a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:40.673299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23d90ee3-864c-4d97-b3b5-cc29a797103a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:40.717997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-23d90ee3-864c-4d97-b3b5-cc29a797103a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.071582 Sep 08 14:06:40.718638 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1334/2380] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:40 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:40.723460 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7c6316e1-1020-4027-8fa1-13d59716b6d3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:40.726305 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7c6316e1-1020-4027-8fa1-13d59716b6d3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:40.733635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7c6316e1-1020-4027-8fa1-13d59716b6d3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:40.734176 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7c6316e1-1020-4027-8fa1-13d59716b6d3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:40.734597 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7c6316e1-1020-4027-8fa1-13d59716b6d3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:40.735298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7c6316e1-1020-4027-8fa1-13d59716b6d3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:40.735755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7c6316e1-1020-4027-8fa1-13d59716b6d3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:40.736184 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7c6316e1-1020-4027-8fa1-13d59716b6d3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:40.775579 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7c6316e1-1020-4027-8fa1-13d59716b6d3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055856 Sep 08 14:06:40.776170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1335/2381] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:40 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:41.562055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3f92be63-0f6e-4a28-bfb3-3b9d8b864c6f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:41.562854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3f92be63-0f6e-4a28-bfb3-3b9d8b864c6f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:41.570716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3f92be63-0f6e-4a28-bfb3-3b9d8b864c6f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:41.571669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3f92be63-0f6e-4a28-bfb3-3b9d8b864c6f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:41.571669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3f92be63-0f6e-4a28-bfb3-3b9d8b864c6f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:41.572303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3f92be63-0f6e-4a28-bfb3-3b9d8b864c6f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:41.572761 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3f92be63-0f6e-4a28-bfb3-3b9d8b864c6f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:41.573183 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3f92be63-0f6e-4a28-bfb3-3b9d8b864c6f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:41.749244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3f92be63-0f6e-4a28-bfb3-3b9d8b864c6f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:41.758862 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3f92be63-0f6e-4a28-bfb3-3b9d8b864c6f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.201102 Sep 08 14:06:41.759331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1336/2382] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:41 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:41.763829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3e9ab934-10dc-4b85-94db-cd138d7b5bd5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:41.764576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3e9ab934-10dc-4b85-94db-cd138d7b5bd5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:41.772293 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3e9ab934-10dc-4b85-94db-cd138d7b5bd5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:41.772940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3e9ab934-10dc-4b85-94db-cd138d7b5bd5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:41.773581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3e9ab934-10dc-4b85-94db-cd138d7b5bd5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:41.774430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3e9ab934-10dc-4b85-94db-cd138d7b5bd5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:41.774993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3e9ab934-10dc-4b85-94db-cd138d7b5bd5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:41.775616 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3e9ab934-10dc-4b85-94db-cd138d7b5bd5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:41.848261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3e9ab934-10dc-4b85-94db-cd138d7b5bd5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:41.859802 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3e9ab934-10dc-4b85-94db-cd138d7b5bd5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.099454 Sep 08 14:06:41.860385 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1337/2383] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:41 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:41.865593 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a2be2cc8-38ce-4c34-9228-9deb6f9ede39 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:06:41.867965 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a2be2cc8-38ce-4c34-9228-9deb6f9ede39 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:41.875048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a2be2cc8-38ce-4c34-9228-9deb6f9ede39 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:41.887644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a2be2cc8-38ce-4c34-9228-9deb6f9ede39 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:41.887644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a2be2cc8-38ce-4c34-9228-9deb6f9ede39 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:41.887644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a2be2cc8-38ce-4c34-9228-9deb6f9ede39 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:41.887644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a2be2cc8-38ce-4c34-9228-9deb6f9ede39 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:41.888387 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a2be2cc8-38ce-4c34-9228-9deb6f9ede39 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:42.016697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a2be2cc8-38ce-4c34-9228-9deb6f9ede39 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "POST /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/action" status: 200 len: 76038 microversion: 2.1 time: 0.154989 Sep 08 14:06:42.017453 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1338/2384] 10.222.0.25 () {60 vars in 1278 bytes} [Thu Sep 8 14:06:41 2022] POST /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152/action => generated 76038 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:06:42.023620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-dee995a9-e173-4fce-8c3a-b8c967902de2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:42.025942 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-dee995a9-e173-4fce-8c3a-b8c967902de2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:42.034682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dee995a9-e173-4fce-8c3a-b8c967902de2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:42.035681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dee995a9-e173-4fce-8c3a-b8c967902de2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:42.036301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dee995a9-e173-4fce-8c3a-b8c967902de2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:42.037300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dee995a9-e173-4fce-8c3a-b8c967902de2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:42.037854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dee995a9-e173-4fce-8c3a-b8c967902de2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:42.038423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dee995a9-e173-4fce-8c3a-b8c967902de2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:42.074507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-dee995a9-e173-4fce-8c3a-b8c967902de2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055592 Sep 08 14:06:42.075085 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1339/2385] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:42 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:42.796929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fd576f0f-4794-42be-826e-64dd9ff5e663 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:42.797960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fd576f0f-4794-42be-826e-64dd9ff5e663 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:42.800430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:42.806227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd576f0f-4794-42be-826e-64dd9ff5e663 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:42.806777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd576f0f-4794-42be-826e-64dd9ff5e663 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:42.807259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd576f0f-4794-42be-826e-64dd9ff5e663 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:42.808004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd576f0f-4794-42be-826e-64dd9ff5e663 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:42.808551 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd576f0f-4794-42be-826e-64dd9ff5e663 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:42.809040 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fd576f0f-4794-42be-826e-64dd9ff5e663 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:42.995946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fd576f0f-4794-42be-826e-64dd9ff5e663 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:43.008052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fd576f0f-4794-42be-826e-64dd9ff5e663 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.215486 Sep 08 14:06:43.008726 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1340/2386] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:42 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:43.017791 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d189a2a5-0e82-4528-8899-398c79e7cd9e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:43.024474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d189a2a5-0e82-4528-8899-398c79e7cd9e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:43.030521 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d189a2a5-0e82-4528-8899-398c79e7cd9e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:43.031285 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d189a2a5-0e82-4528-8899-398c79e7cd9e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:43.032096 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d189a2a5-0e82-4528-8899-398c79e7cd9e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:43.033403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d189a2a5-0e82-4528-8899-398c79e7cd9e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:43.034288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d189a2a5-0e82-4528-8899-398c79e7cd9e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:43.035068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d189a2a5-0e82-4528-8899-398c79e7cd9e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:43.121207 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d189a2a5-0e82-4528-8899-398c79e7cd9e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:43.136116 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d189a2a5-0e82-4528-8899-398c79e7cd9e tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.125914 Sep 08 14:06:43.136717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1341/2387] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:43 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:43.144549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ba8ce79a-bd15-4a1e-96bb-6ae054263b44 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:43.148328 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ba8ce79a-bd15-4a1e-96bb-6ae054263b44 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:43.158788 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba8ce79a-bd15-4a1e-96bb-6ae054263b44 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:43.159670 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba8ce79a-bd15-4a1e-96bb-6ae054263b44 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:43.160459 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba8ce79a-bd15-4a1e-96bb-6ae054263b44 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:43.161647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba8ce79a-bd15-4a1e-96bb-6ae054263b44 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:43.162458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba8ce79a-bd15-4a1e-96bb-6ae054263b44 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:43.163299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba8ce79a-bd15-4a1e-96bb-6ae054263b44 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:43.204234 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ba8ce79a-bd15-4a1e-96bb-6ae054263b44 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.066261 Sep 08 14:06:43.204722 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1342/2388] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:43 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:44.047223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a92cef08-4730-4d3a-aca2-7a5452a132e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:44.048266 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a92cef08-4730-4d3a-aca2-7a5452a132e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:44.055228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a92cef08-4730-4d3a-aca2-7a5452a132e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:44.055865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a92cef08-4730-4d3a-aca2-7a5452a132e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:44.056299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a92cef08-4730-4d3a-aca2-7a5452a132e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:44.057014 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a92cef08-4730-4d3a-aca2-7a5452a132e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:44.057471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a92cef08-4730-4d3a-aca2-7a5452a132e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:44.057899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a92cef08-4730-4d3a-aca2-7a5452a132e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:44.083811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=100 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:44.260471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a92cef08-4730-4d3a-aca2-7a5452a132e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:44.277709 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a92cef08-4730-4d3a-aca2-7a5452a132e3 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.234244 Sep 08 14:06:44.278958 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1343/2389] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:44 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:06:44.283859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0a7f2e26-b325-4612-a154-e852fcaafe51 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:44.284494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0a7f2e26-b325-4612-a154-e852fcaafe51 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:44.291925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0a7f2e26-b325-4612-a154-e852fcaafe51 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:44.292497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0a7f2e26-b325-4612-a154-e852fcaafe51 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:44.293014 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0a7f2e26-b325-4612-a154-e852fcaafe51 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:44.293792 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0a7f2e26-b325-4612-a154-e852fcaafe51 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:44.294546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0a7f2e26-b325-4612-a154-e852fcaafe51 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:44.295066 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0a7f2e26-b325-4612-a154-e852fcaafe51 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:44.394368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0a7f2e26-b325-4612-a154-e852fcaafe51 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:06:44.404746 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0a7f2e26-b325-4612-a154-e852fcaafe51 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.124884 Sep 08 14:06:44.405340 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1344/2390] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:06:44 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:06:44.410511 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4694622f-88a3-41a8-aafc-68d221ca0df2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:44.413767 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4694622f-88a3-41a8-aafc-68d221ca0df2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:44.421520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4694622f-88a3-41a8-aafc-68d221ca0df2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:44.422170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4694622f-88a3-41a8-aafc-68d221ca0df2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:44.422610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4694622f-88a3-41a8-aafc-68d221ca0df2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:44.423323 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4694622f-88a3-41a8-aafc-68d221ca0df2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:44.423776 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4694622f-88a3-41a8-aafc-68d221ca0df2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:44.424198 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4694622f-88a3-41a8-aafc-68d221ca0df2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:44.460075 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4694622f-88a3-41a8-aafc-68d221ca0df2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053602 Sep 08 14:06:44.460789 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1345/2391] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:06:44 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:06:45.291527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-292b4048-cf2f-4297-b350-38b471bb9f8f tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:06:45.292409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-292b4048-cf2f-4297-b350-38b471bb9f8f tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:06:45.299199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-292b4048-cf2f-4297-b350-38b471bb9f8f tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:45.299687 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-292b4048-cf2f-4297-b350-38b471bb9f8f tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:45.300131 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-292b4048-cf2f-4297-b350-38b471bb9f8f tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:45.301023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-292b4048-cf2f-4297-b350-38b471bb9f8f tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:06:45.301520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-292b4048-cf2f-4297-b350-38b471bb9f8f tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:06:45.301950 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-292b4048-cf2f-4297-b350-38b471bb9f8f tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:06:55.295828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=13 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:55.367749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=19 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:16.227199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:07:16.227839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:07:16 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:07:34.676093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:07:34.676911 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:07:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:07:45.282892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-55e3f370-f4b9-4387-93a9-8453ffc55833 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lazy-loading 'services' on Instance uuid 7ec70561-b436-4d03-bd60-6c5ae232b492 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:07:45.462957 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-292b4048-cf2f-4297-b350-38b471bb9f8f tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lazy-loading 'services' on Instance uuid 7ec70561-b436-4d03-bd60-6c5ae232b492 {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:07:55.287766 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=4 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:55.498892 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=4 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:47.529093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:08:47.529841 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:08:47 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:08:50.933721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:08:50.934326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:08:50 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:09:03.578856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:09:03.584242 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:09:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:09:46.068536 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-55e3f370-f4b9-4387-93a9-8453ffc55833 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70" status: 202 len: 0 microversion: 2.1 time: 240.938270 Sep 08 14:09:46.070083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1047/2392] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:05:45 2022] DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 => generated 0 bytes in 240941 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:09:46.079153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-292b4048-cf2f-4297-b350-38b471bb9f8f tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5cc8837f-aeb5-48bf-aeaa-a52cf19ddd94) Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-292b4048-cf2f-4297-b350-38b471bb9f8f tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Request method failure captured: Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 HTTP/1.1 Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfSNyJo8zI7plzmoT3joT1LB7boaKVSoM2Btn54mkWpdqceI6SLQFx4_Rpb5l-mqzqX661NrDxpQinv-CjzSmbNBVW650b02MoLz_tOe3RLJyylDgcU4AMsS5LTkC_9jOXC-JdkvqUTGjOl36FG-vYox8h_LoJX8tIcUOcoYc5HdH0s Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:09:46.083548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:09:46.086819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:09:46.086819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:09:46.086819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:09:46.086819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:09:46.086819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:09:46.086819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:09:46.086819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:09:46.086819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 5ae9377cf36e4086aa38c07b021fdead Sep 08 14:09:46.086819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:09:46.086819 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:09:46.088302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5cc8837f-aeb5-48bf-aeaa-a52cf19ddd94) Sep 08 14:09:46.088302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'server_id': '7ec70561-b436-4d03-bd60-6c5ae232b492', 'id': 'fa5de96f-82e6-44dd-a208-403b6087dd70'} Sep 08 14:09:46.088302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:09:46.088302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:09:46.088302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:09:46.088302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:09:46.088302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:09:46.088302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:09:46.088302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:09:46.088302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:09:46.088302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:09:46.088302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:09:46.088302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:09:46.088302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:09:46.088302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:09:46.088302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:09:46.088302 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5cc8837f-aeb5-48bf-aeaa-a52cf19ddd94) Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:09:46.090960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:09:46.093659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:09:46.093659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:09:46.093659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:09:46.093659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:09:46.093659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:09:46.093659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:09:46.093659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:09:46.093659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:09:46.093659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:09:46.093659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:09:46.093659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:09:46.093659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5cc8837f-aeb5-48bf-aeaa-a52cf19ddd94) Sep 08 14:09:46.093659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:09:46.093659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:09:46.093659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:09:46.093659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:09:46.093659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 08 14:09:46.097505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 08 14:09:46.097505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 14:09:46.097505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:09:46.097505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 08 14:09:46.097505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 08 14:09:46.097505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 08 14:09:46.097505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 08 14:09:46.097505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 08 14:09:46.097505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:09:46.097505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 08 14:09:46.097505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 08 14:09:46.097505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5cc8837f-aeb5-48bf-aeaa-a52cf19ddd94) Sep 08 14:09:46.097505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:09:46.097505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:09:46.097505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:09:46.097505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:09:46.097505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:09:46.097505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:09:46.101161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:09:46.101161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:09:46.101161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:09:46.101161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:09:46.101161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 08 14:09:46.101161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 08 14:09:46.101161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5cc8837f-aeb5-48bf-aeaa-a52cf19ddd94) Sep 08 14:09:46.101161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:09:46.101161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:09:46.101161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:09:46.101161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:09:46.101161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:09:46.101161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:09:46.101161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:09:46.101161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:09:46.101161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5cc8837f-aeb5-48bf-aeaa-a52cf19ddd94) Sep 08 14:09:46.101161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:09:46.107439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-292b4048-cf2f-4297-b350-38b471bb9f8f tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-5cc8837f-aeb5-48bf-aeaa-a52cf19ddd94) {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:09:46.107439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-292b4048-cf2f-4297-b350-38b471bb9f8f tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70" status: 400 len: 269 microversion: 2.1 time: 180.893395 Sep 08 14:09:46.107439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1346/2393] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:06:45 2022] DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 => generated 269 bytes in 180896 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 08 14:09:46.108444 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-44ec1cc1-ffa5-416c-bad3-34c9f410e0fd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:09:46.108444 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-44ec1cc1-ffa5-416c-bad3-34c9f410e0fd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:09:46.108854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a3be4b71-05e2-4fb1-a761-ff583a0e4d92 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:09:46.108854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a3be4b71-05e2-4fb1-a761-ff583a0e4d92 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:09:46.113618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-44ec1cc1-ffa5-416c-bad3-34c9f410e0fd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:09:46.115764 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-44ec1cc1-ffa5-416c-bad3-34c9f410e0fd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:09:46.116450 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-44ec1cc1-ffa5-416c-bad3-34c9f410e0fd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:09:46.117741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-44ec1cc1-ffa5-416c-bad3-34c9f410e0fd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:09:46.118627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-44ec1cc1-ffa5-416c-bad3-34c9f410e0fd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:09:46.119572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-44ec1cc1-ffa5-416c-bad3-34c9f410e0fd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:09:46.122054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a3be4b71-05e2-4fb1-a761-ff583a0e4d92 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:09:46.122688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a3be4b71-05e2-4fb1-a761-ff583a0e4d92 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:09:46.123219 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a3be4b71-05e2-4fb1-a761-ff583a0e4d92 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:09:46.124746 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a3be4b71-05e2-4fb1-a761-ff583a0e4d92 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:09:46.125933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a3be4b71-05e2-4fb1-a761-ff583a0e4d92 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:09:46.126500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a3be4b71-05e2-4fb1-a761-ff583a0e4d92 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:09:46.244137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a3be4b71-05e2-4fb1-a761-ff583a0e4d92 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:09:46.254020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a3be4b71-05e2-4fb1-a761-ff583a0e4d92 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.164616 Sep 08 14:09:46.254695 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1347/2394] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:09:46 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:09:46.346666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-44ec1cc1-ffa5-416c-bad3-34c9f410e0fd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:09:46.349346 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bd1f200e-4422-4eb5-a764-7a538a34729f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:09:46.358742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bd1f200e-4422-4eb5-a764-7a538a34729f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:09:46.358851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-44ec1cc1-ffa5-416c-bad3-34c9f410e0fd tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.284604 Sep 08 14:09:46.358851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1048/2395] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:09:46 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 286 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:09:46.361376 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd1f200e-4422-4eb5-a764-7a538a34729f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:09:46.362245 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-27b11881-8854-4814-80db-bbbc97ba50b1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:09:46.362487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd1f200e-4422-4eb5-a764-7a538a34729f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:09:46.362931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-27b11881-8854-4814-80db-bbbc97ba50b1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:09:46.363398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd1f200e-4422-4eb5-a764-7a538a34729f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:09:46.364673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd1f200e-4422-4eb5-a764-7a538a34729f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:09:46.366057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd1f200e-4422-4eb5-a764-7a538a34729f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:09:46.366880 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bd1f200e-4422-4eb5-a764-7a538a34729f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:09:46.369600 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27b11881-8854-4814-80db-bbbc97ba50b1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:09:46.370389 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27b11881-8854-4814-80db-bbbc97ba50b1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:09:46.371029 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27b11881-8854-4814-80db-bbbc97ba50b1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:09:46.372095 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27b11881-8854-4814-80db-bbbc97ba50b1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:09:46.372723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27b11881-8854-4814-80db-bbbc97ba50b1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:09:46.373309 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27b11881-8854-4814-80db-bbbc97ba50b1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:09:46.405555 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bd1f200e-4422-4eb5-a764-7a538a34729f tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.148754 Sep 08 14:09:46.406165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1348/2396] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:09:46 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:09:46.411926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-425f9558-283a-4f6b-9893-0b42d5d4f180 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:09:46.412639 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-425f9558-283a-4f6b-9893-0b42d5d4f180 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:09:46.419292 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-425f9558-283a-4f6b-9893-0b42d5d4f180 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:09:46.421206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-425f9558-283a-4f6b-9893-0b42d5d4f180 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:09:46.421206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-425f9558-283a-4f6b-9893-0b42d5d4f180 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:09:46.421416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-425f9558-283a-4f6b-9893-0b42d5d4f180 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:09:46.421734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-425f9558-283a-4f6b-9893-0b42d5d4f180 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:09:46.422988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-425f9558-283a-4f6b-9893-0b42d5d4f180 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:09:46.626648 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-425f9558-283a-4f6b-9893-0b42d5d4f180 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:09:46.639122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-425f9558-283a-4f6b-9893-0b42d5d4f180 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.231176 Sep 08 14:09:46.640483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1349/2397] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:09:46 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:09:46.644864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6e4461b9-a54d-470d-b92c-818196e919c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:09:46.645471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6e4461b9-a54d-470d-b92c-818196e919c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:09:46.651921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6e4461b9-a54d-470d-b92c-818196e919c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:09:46.652445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6e4461b9-a54d-470d-b92c-818196e919c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:09:46.652856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6e4461b9-a54d-470d-b92c-818196e919c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:09:46.653555 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6e4461b9-a54d-470d-b92c-818196e919c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:09:46.654011 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6e4461b9-a54d-470d-b92c-818196e919c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:09:46.654471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6e4461b9-a54d-470d-b92c-818196e919c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:09:46.736197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6e4461b9-a54d-470d-b92c-818196e919c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:09:46.746515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6e4461b9-a54d-470d-b92c-818196e919c5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.105241 Sep 08 14:09:46.747444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1350/2398] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:09:46 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:09:46.893088 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-262750ea-1f48-4a8a-99ad-f79df0e35fc5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:09:46.893575 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-262750ea-1f48-4a8a-99ad-f79df0e35fc5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:09:46.899703 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-262750ea-1f48-4a8a-99ad-f79df0e35fc5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:09:46.900183 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-262750ea-1f48-4a8a-99ad-f79df0e35fc5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:09:46.900614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-262750ea-1f48-4a8a-99ad-f79df0e35fc5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:09:46.901395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-262750ea-1f48-4a8a-99ad-f79df0e35fc5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:09:46.901843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-262750ea-1f48-4a8a-99ad-f79df0e35fc5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:09:46.902288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-262750ea-1f48-4a8a-99ad-f79df0e35fc5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:09:46.930818 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-262750ea-1f48-4a8a-99ad-f79df0e35fc5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:09:47.362120 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-262750ea-1f48-4a8a-99ad-f79df0e35fc5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 204 len: 0 microversion: 2.1 time: 0.613007 Sep 08 14:09:47.362847 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1351/2399] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:09:46 2022] DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:09:47.368940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b0b47078-0990-4606-8a6f-36fff8ad7eec tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:09:47.371506 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b0b47078-0990-4606-8a6f-36fff8ad7eec tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:09:47.378484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b0b47078-0990-4606-8a6f-36fff8ad7eec tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:09:47.378988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b0b47078-0990-4606-8a6f-36fff8ad7eec tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:09:47.379427 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b0b47078-0990-4606-8a6f-36fff8ad7eec tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:09:47.380139 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b0b47078-0990-4606-8a6f-36fff8ad7eec tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:09:47.380603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b0b47078-0990-4606-8a6f-36fff8ad7eec tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:09:47.381401 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b0b47078-0990-4606-8a6f-36fff8ad7eec tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:09:47.413898 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b0b47078-0990-4606-8a6f-36fff8ad7eec tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049652 Sep 08 14:09:47.416158 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1352/2400] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:09:47 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:09:47.419337 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e422c68c-f4b3-47b6-8953-9133649431ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:09:47.419816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e422c68c-f4b3-47b6-8953-9133649431ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:09:47.426046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e422c68c-f4b3-47b6-8953-9133649431ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:09:47.426772 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e422c68c-f4b3-47b6-8953-9133649431ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:09:47.427204 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e422c68c-f4b3-47b6-8953-9133649431ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:09:47.427895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e422c68c-f4b3-47b6-8953-9133649431ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:09:47.428429 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e422c68c-f4b3-47b6-8953-9133649431ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:09:47.428857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e422c68c-f4b3-47b6-8953-9133649431ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:09:56.369798 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=4 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:56.399318 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=11,nova_cell1:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:57.424138 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=19,nova_api:UPDATE=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:57.450699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=48,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:36.240039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:10:36.241226 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:10:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:11:46.761271 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-27b11881-8854-4814-80db-bbbc97ba50b1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lazy-loading 'services' on Instance uuid 7ec70561-b436-4d03-bd60-6c5ae232b492 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:11:46.806671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-e422c68c-f4b3-47b6-8953-9133649431ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lazy-loading 'services' on Instance uuid 7ec70561-b436-4d03-bd60-6c5ae232b492 {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:11:56.765435 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=4 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:56.810943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=4 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:12:07.541173 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:12:07.541875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:12:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:12:10.945053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:12:10.945621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:12:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:12:23.589489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:12:23.590188 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:12:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:12:34.688468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:12:34.689134 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:12:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:13:31.581428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-27b11881-8854-4814-80db-bbbc97ba50b1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-bad9748f-a1dd-486d-9c0b-14d062e404fa) Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-27b11881-8854-4814-80db-bbbc97ba50b1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Request method failure captured: Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 HTTP/1.1 Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfSNyJo8zI7plzmoT3joT1LB7boaKVSoM2Btn54mkWpdqceI6SLQFx4_Rpb5l-mqzqX661NrDxpQinv-CjzSmbNBVW650b02MoLz_tOe3RLJyylDgcU4AMsS5LTkC_9jOXC-JdkvqUTGjOl36FG-vYox8h_LoJX8tIcUOcoYc5HdH0s Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:13:31.583792 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:13:31.585296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:13:31.585296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:13:31.585296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:13:31.585296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:13:31.585296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:13:31.585296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:13:31.585296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:13:31.585296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 5ae9377cf36e4086aa38c07b021fdead Sep 08 14:13:31.585296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:13:31.585296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:13:31.586282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-bad9748f-a1dd-486d-9c0b-14d062e404fa) Sep 08 14:13:31.586282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'server_id': '7ec70561-b436-4d03-bd60-6c5ae232b492', 'id': 'fa5de96f-82e6-44dd-a208-403b6087dd70'} Sep 08 14:13:31.586282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:13:31.586282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:13:31.586282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:13:31.586282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:13:31.586282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:13:31.586282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:13:31.586282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:13:31.586282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:13:31.586282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:13:31.586282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:13:31.586282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:13:31.586282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:13:31.586282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:13:31.586282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:13:31.586282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-bad9748f-a1dd-486d-9c0b-14d062e404fa) Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:13:31.587491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:13:31.589505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:13:31.589505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:13:31.589505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:13:31.589505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:13:31.589505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:13:31.589505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:13:31.589505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:13:31.589505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:13:31.589505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:13:31.589505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:13:31.589505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:13:31.589505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-bad9748f-a1dd-486d-9c0b-14d062e404fa) Sep 08 14:13:31.589505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.589505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:13:31.589505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.589505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:13:31.589505 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 08 14:13:31.591468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 08 14:13:31.591468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 14:13:31.591468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:13:31.591468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 08 14:13:31.591468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 08 14:13:31.591468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 08 14:13:31.591468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 08 14:13:31.591468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 08 14:13:31.591468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:13:31.591468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 08 14:13:31.591468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 08 14:13:31.591468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-bad9748f-a1dd-486d-9c0b-14d062e404fa) Sep 08 14:13:31.591468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.591468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:13:31.591468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.591468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:13:31.591468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:13:31.591468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:13:31.593122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:13:31.593122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:13:31.593122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:13:31.593122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:13:31.593122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 08 14:13:31.593122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 08 14:13:31.593122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-bad9748f-a1dd-486d-9c0b-14d062e404fa) Sep 08 14:13:31.593122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.593122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:13:31.593122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.593122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:13:31.593122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:13:31.593122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:13:31.593122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:13:31.593122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:13:31.593122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-bad9748f-a1dd-486d-9c0b-14d062e404fa) Sep 08 14:13:31.593122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.595862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-27b11881-8854-4814-80db-bbbc97ba50b1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-bad9748f-a1dd-486d-9c0b-14d062e404fa) {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:13:31.595862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-27b11881-8854-4814-80db-bbbc97ba50b1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70" status: 400 len: 269 microversion: 2.1 time: 225.227355 Sep 08 14:13:31.595862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:13:31 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:13:31.595862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:13:31 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (10.222.0.25) Sep 08 14:13:31.595862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-27b11881-8854-4814-80db-bbbc97ba50b1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Unhandled error: OSError: write error Sep 08 14:13:31.595862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:13:31.595862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:13:31.597491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1049/2401] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:09:46 2022] DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 => generated 0 bytes in 225229 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:13:31.621475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-e422c68c-f4b3-47b6-8953-9133649431ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-88997f6f-21a9-44ab-854c-4c090159d789) Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e422c68c-f4b3-47b6-8953-9133649431ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Request method failure captured: Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 HTTP/1.1 Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfSNyJo8zI7plzmoT3joT1LB7boaKVSoM2Btn54mkWpdqceI6SLQFx4_Rpb5l-mqzqX661NrDxpQinv-CjzSmbNBVW650b02MoLz_tOe3RLJyylDgcU4AMsS5LTkC_9jOXC-JdkvqUTGjOl36FG-vYox8h_LoJX8tIcUOcoYc5HdH0s Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:13:31.627935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:13:31.630558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:13:31.630558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:13:31.630558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:13:31.630558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:13:31.630558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:13:31.630558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:13:31.630558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:13:31.630558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 5ae9377cf36e4086aa38c07b021fdead Sep 08 14:13:31.630558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:13:31.630558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:13:31.631728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-88997f6f-21a9-44ab-854c-4c090159d789) Sep 08 14:13:31.631728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'server_id': '7ec70561-b436-4d03-bd60-6c5ae232b492', 'id': 'fa5de96f-82e6-44dd-a208-403b6087dd70'} Sep 08 14:13:31.631728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:13:31.631728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:13:31.631728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:13:31.631728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:13:31.631728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:13:31.631728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:13:31.631728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:13:31.631728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:13:31.631728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:13:31.631728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:13:31.631728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:13:31.631728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:13:31.631728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:13:31.631728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:13:31.631728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-88997f6f-21a9-44ab-854c-4c090159d789) Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:13:31.633674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:13:31.636512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:13:31.636512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:13:31.636512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:13:31.636512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:13:31.636512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:13:31.636512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:13:31.636512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:13:31.636512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:13:31.636512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:13:31.636512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:13:31.636512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:13:31.636512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-88997f6f-21a9-44ab-854c-4c090159d789) Sep 08 14:13:31.636512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.636512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:13:31.636512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.636512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:13:31.636512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 08 14:13:31.638913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 08 14:13:31.638913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 14:13:31.638913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:13:31.638913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 08 14:13:31.638913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 08 14:13:31.638913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 08 14:13:31.638913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 08 14:13:31.638913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 08 14:13:31.638913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:13:31.638913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 08 14:13:31.638913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 08 14:13:31.638913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-88997f6f-21a9-44ab-854c-4c090159d789) Sep 08 14:13:31.638913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.638913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:13:31.638913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.638913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:13:31.638913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:13:31.638913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:13:31.641149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:13:31.641149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:13:31.641149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:13:31.641149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:13:31.641149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 08 14:13:31.641149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 08 14:13:31.641149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-88997f6f-21a9-44ab-854c-4c090159d789) Sep 08 14:13:31.641149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.641149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:13:31.641149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.641149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:13:31.641149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:13:31.641149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:13:31.641149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:13:31.641149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:13:31.641149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-88997f6f-21a9-44ab-854c-4c090159d789) Sep 08 14:13:31.641149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.646567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e422c68c-f4b3-47b6-8953-9133649431ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-88997f6f-21a9-44ab-854c-4c090159d789) {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:13:31.648003 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e422c68c-f4b3-47b6-8953-9133649431ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70" status: 400 len: 269 microversion: 2.1 time: 224.232156 Sep 08 14:13:31.654175 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1353/2402] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:09:47 2022] DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 => generated 269 bytes in 224234 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 08 14:13:31.654689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f276c42c-366b-43e1-ba00-5b3aa8a9e320 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:13:31.656697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f276c42c-366b-43e1-ba00-5b3aa8a9e320 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:13:31.666364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f276c42c-366b-43e1-ba00-5b3aa8a9e320 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:31.666364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f276c42c-366b-43e1-ba00-5b3aa8a9e320 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:31.666364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f276c42c-366b-43e1-ba00-5b3aa8a9e320 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:31.666364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f276c42c-366b-43e1-ba00-5b3aa8a9e320 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:31.667236 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f276c42c-366b-43e1-ba00-5b3aa8a9e320 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:31.667669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f276c42c-366b-43e1-ba00-5b3aa8a9e320 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:31.745951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f276c42c-366b-43e1-ba00-5b3aa8a9e320 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:13:31.756094 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f276c42c-366b-43e1-ba00-5b3aa8a9e320 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.105955 Sep 08 14:13:31.756680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1354/2403] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:13:31 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:13:31.763254 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-aed511d5-392a-48f7-8068-6c87e89045d2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:13:31.770925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-aed511d5-392a-48f7-8068-6c87e89045d2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:13:31.772408 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aed511d5-392a-48f7-8068-6c87e89045d2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:31.772894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aed511d5-392a-48f7-8068-6c87e89045d2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:31.773330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aed511d5-392a-48f7-8068-6c87e89045d2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:31.774036 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aed511d5-392a-48f7-8068-6c87e89045d2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:31.774503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aed511d5-392a-48f7-8068-6c87e89045d2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:31.774927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aed511d5-392a-48f7-8068-6c87e89045d2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:31.807809 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-aed511d5-392a-48f7-8068-6c87e89045d2 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050033 Sep 08 14:13:31.808345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1355/2404] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:13:31 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:13:31.811272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5306ede7-0a52-4e0e-af24-bee064d51c95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:13:31.811978 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5306ede7-0a52-4e0e-af24-bee064d51c95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:13:31.812876 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-edecaadb-9e06-4c11-8133-ccdc42284755 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:13:31.813365 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-edecaadb-9e06-4c11-8133-ccdc42284755 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:13:31.820657 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5306ede7-0a52-4e0e-af24-bee064d51c95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:31.828585 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-edecaadb-9e06-4c11-8133-ccdc42284755 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:31.828772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5306ede7-0a52-4e0e-af24-bee064d51c95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:31.828900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-edecaadb-9e06-4c11-8133-ccdc42284755 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:31.828900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-edecaadb-9e06-4c11-8133-ccdc42284755 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:31.829101 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5306ede7-0a52-4e0e-af24-bee064d51c95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:31.829203 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-edecaadb-9e06-4c11-8133-ccdc42284755 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:31.829302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5306ede7-0a52-4e0e-af24-bee064d51c95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:31.829302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5306ede7-0a52-4e0e-af24-bee064d51c95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:31.829499 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-edecaadb-9e06-4c11-8133-ccdc42284755 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:31.829595 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5306ede7-0a52-4e0e-af24-bee064d51c95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:31.829687 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-edecaadb-9e06-4c11-8133-ccdc42284755 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:31.839305 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-edecaadb-9e06-4c11-8133-ccdc42284755 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-edecaadb-9e06-4c11-8133-ccdc42284755 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 HTTP/1.1 Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:13:31.840897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:13:31.843489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:13:31.843489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:13:31.843489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:13:31.843489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:13:31.843489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:13:31.843489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:13:31.843489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:13:31.843489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:13:31.843489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:13:31.843489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:13:31.843489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:13:31.843489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': '5a9f0e69-51d0-4a26-89d2-91c47d197556'} Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:13:31.845124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:13:31.846814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:13:31.848135 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-edecaadb-9e06-4c11-8133-ccdc42284755 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:13:31.848135 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-edecaadb-9e06-4c11-8133-ccdc42284755 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 404 len: 111 microversion: 2.1 time: 0.032983 Sep 08 14:13:31.848135 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1356/2405] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:13:31 2022] DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:13:31.848420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-54b9d868-f8e8-4d3c-be82-c7395c700551 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:13:31.848926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-54b9d868-f8e8-4d3c-be82-c7395c700551 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:13:31.855767 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-54b9d868-f8e8-4d3c-be82-c7395c700551 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:31.856234 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-54b9d868-f8e8-4d3c-be82-c7395c700551 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:31.856710 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-54b9d868-f8e8-4d3c-be82-c7395c700551 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:31.857482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-54b9d868-f8e8-4d3c-be82-c7395c700551 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:31.857814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-54b9d868-f8e8-4d3c-be82-c7395c700551 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:31.858348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-54b9d868-f8e8-4d3c-be82-c7395c700551 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:32.002620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5306ede7-0a52-4e0e-af24-bee064d51c95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:13:32.011943 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5306ede7-0a52-4e0e-af24-bee064d51c95 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.424291 Sep 08 14:13:32.019858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1050/2406] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:13:31 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 425 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:13:32.019858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5328b484-0050-46cd-bdc4-be422ddfd1d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:13:32.019858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5328b484-0050-46cd-bdc4-be422ddfd1d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:13:32.024267 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5328b484-0050-46cd-bdc4-be422ddfd1d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:32.024761 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5328b484-0050-46cd-bdc4-be422ddfd1d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:32.025185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5328b484-0050-46cd-bdc4-be422ddfd1d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:32.025913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5328b484-0050-46cd-bdc4-be422ddfd1d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:32.026417 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5328b484-0050-46cd-bdc4-be422ddfd1d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:32.026869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5328b484-0050-46cd-bdc4-be422ddfd1d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:32.172728 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5328b484-0050-46cd-bdc4-be422ddfd1d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:13:32.184778 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5328b484-0050-46cd-bdc4-be422ddfd1d6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.171151 Sep 08 14:13:32.185377 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1051/2407] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:13:32 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:13:32.190780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9ad41812-bb3f-44e2-b360-507edf843ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:13:32.191373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9ad41812-bb3f-44e2-b360-507edf843ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:13:32.197455 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9ad41812-bb3f-44e2-b360-507edf843ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:32.197938 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9ad41812-bb3f-44e2-b360-507edf843ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:32.198460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9ad41812-bb3f-44e2-b360-507edf843ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:32.199093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9ad41812-bb3f-44e2-b360-507edf843ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:32.199545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9ad41812-bb3f-44e2-b360-507edf843ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:32.199956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9ad41812-bb3f-44e2-b360-507edf843ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:32.272755 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9ad41812-bb3f-44e2-b360-507edf843ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '772205d93ce047e9ad4ddefadec300f4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '45579d93010d4ed0b80cad3a8b85ea50', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:13:32.282555 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9ad41812-bb3f-44e2-b360-507edf843ed2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 200 len: 1224 microversion: 2.1 time: 0.096040 Sep 08 14:13:32.283144 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1052/2408] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:13:32 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 1224 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:13:32.287824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b1601446-a717-49fe-a0f4-b3f2ea86da6f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:13:32.288316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b1601446-a717-49fe-a0f4-b3f2ea86da6f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:13:32.296044 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1601446-a717-49fe-a0f4-b3f2ea86da6f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:32.296537 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1601446-a717-49fe-a0f4-b3f2ea86da6f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:32.296968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1601446-a717-49fe-a0f4-b3f2ea86da6f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:32.297670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1601446-a717-49fe-a0f4-b3f2ea86da6f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:32.298119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1601446-a717-49fe-a0f4-b3f2ea86da6f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:32.299668 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1601446-a717-49fe-a0f4-b3f2ea86da6f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:32.314216 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-b1601446-a717-49fe-a0f4-b3f2ea86da6f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b1601446-a717-49fe-a0f4-b3f2ea86da6f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 HTTP/1.1 Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:13:32.316542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:13:32.318460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:13:32.318460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:13:32.318460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:13:32.318460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:13:32.318460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:13:32.318460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:13:32.318460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:13:32.318460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:13:32.318460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:13:32.318460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:13:32.318460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:13:32.318460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '5a9f0e69-51d0-4a26-89d2-91c47d197556'} Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:13:32.319430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:13:32.320796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:13:32.322298 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b1601446-a717-49fe-a0f4-b3f2ea86da6f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:13:32.322298 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b1601446-a717-49fe-a0f4-b3f2ea86da6f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 404 len: 111 microversion: 2.1 time: 0.033821 Sep 08 14:13:32.322298 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1053/2409] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:13:32 2022] DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:13:32.323419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-27d41c46-e73e-4a3a-813f-f37416717475 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:13:32.325667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-27d41c46-e73e-4a3a-813f-f37416717475 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:13:32.332322 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27d41c46-e73e-4a3a-813f-f37416717475 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:32.332807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27d41c46-e73e-4a3a-813f-f37416717475 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:32.333241 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27d41c46-e73e-4a3a-813f-f37416717475 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:32.333946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27d41c46-e73e-4a3a-813f-f37416717475 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:32.334427 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27d41c46-e73e-4a3a-813f-f37416717475 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:32.334847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-27d41c46-e73e-4a3a-813f-f37416717475 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:32.368350 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-27d41c46-e73e-4a3a-813f-f37416717475 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048832 Sep 08 14:13:32.368929 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1054/2410] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:13:32 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:13:32.433751 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-bf11fe42-5d11-450c-9f42-e29dbb1f8061 req-3fea0099-59f9-4506-a5ca-cc71d7bbf472 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5a9f0e69-51d0-4a26-89d2-91c47d197556", "name": "network-vif-deleted", "tag": "afc328f7-6869-4252-8062-09c20950ee2a"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:13:32.443974 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-bf11fe42-5d11-450c-9f42-e29dbb1f8061 req-3fea0099-59f9-4506-a5ca-cc71d7bbf472 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:32.444555 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-bf11fe42-5d11-450c-9f42-e29dbb1f8061 req-3fea0099-59f9-4506-a5ca-cc71d7bbf472 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:32.444863 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-bf11fe42-5d11-450c-9f42-e29dbb1f8061 req-3fea0099-59f9-4506-a5ca-cc71d7bbf472 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:32.459109 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-bf11fe42-5d11-450c-9f42-e29dbb1f8061 req-3fea0099-59f9-4506-a5ca-cc71d7bbf472 service nova] Creating event network-vif-deleted:afc328f7-6869-4252-8062-09c20950ee2a for instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 on hv-cinder-85646 Sep 08 14:13:32.465976 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-bf11fe42-5d11-450c-9f42-e29dbb1f8061 req-3fea0099-59f9-4506-a5ca-cc71d7bbf472 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.094807 Sep 08 14:13:32.471406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1055/2411] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:13:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:13:32.473782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-63908f62-e5e5-4d45-a7de-c35ba557d897 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:13:32.474457 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-63908f62-e5e5-4d45-a7de-c35ba557d897 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:13:32.481161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-63908f62-e5e5-4d45-a7de-c35ba557d897 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:32.481639 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-63908f62-e5e5-4d45-a7de-c35ba557d897 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:32.482067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-63908f62-e5e5-4d45-a7de-c35ba557d897 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:32.482803 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-63908f62-e5e5-4d45-a7de-c35ba557d897 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:13:32.483254 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-63908f62-e5e5-4d45-a7de-c35ba557d897 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:13:32.483668 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-63908f62-e5e5-4d45-a7de-c35ba557d897 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:13:41.853097 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=8 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:13:41.882061 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=18 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:13:42.477897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=14 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:13:42.508211 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=32 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:13:56.252716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:13:56.253524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:13:56 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:15:27.550687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:15:27.551433 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:15:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:15:30.954989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:15:30.955565 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:15:30 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:15:31.840460 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-54b9d868-f8e8-4d3c-be82-c7395c700551 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lazy-loading 'services' on Instance uuid 7ec70561-b436-4d03-bd60-6c5ae232b492 {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:15:31.934793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-63908f62-e5e5-4d45-a7de-c35ba557d897 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lazy-loading 'services' on Instance uuid 7ec70561-b436-4d03-bd60-6c5ae232b492 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:15:41.845157 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=4 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:41.939576 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=4 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:43.604189 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:15:43.606739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:15:43 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:15:47.189129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-54b9d868-f8e8-4d3c-be82-c7395c700551 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9c24f251-73fa-4232-b72d-622f1d96d749) Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-54b9d868-f8e8-4d3c-be82-c7395c700551 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Request method failure captured: Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 HTTP/1.1 Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfSNyJo8zI7plzmoT3joT1LB7boaKVSoM2Btn54mkWpdqceI6SLQFx4_Rpb5l-mqzqX661NrDxpQinv-CjzSmbNBVW650b02MoLz_tOe3RLJyylDgcU4AMsS5LTkC_9jOXC-JdkvqUTGjOl36FG-vYox8h_LoJX8tIcUOcoYc5HdH0s Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:15:47.191354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:15:47.193275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:15:47.193275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:15:47.193275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:15:47.193275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:15:47.193275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:15:47.193275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:15:47.193275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:15:47.193275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 5ae9377cf36e4086aa38c07b021fdead Sep 08 14:15:47.193275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:15:47.193275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:15:47.194542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9c24f251-73fa-4232-b72d-622f1d96d749) Sep 08 14:15:47.194542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'server_id': '7ec70561-b436-4d03-bd60-6c5ae232b492', 'id': 'fa5de96f-82e6-44dd-a208-403b6087dd70'} Sep 08 14:15:47.194542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:15:47.194542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.194542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:15:47.194542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:15:47.194542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:15:47.194542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:15:47.194542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:15:47.194542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:15:47.194542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:15:47.194542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:15:47.194542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:15:47.194542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:15:47.194542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:15:47.194542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:15:47.194542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9c24f251-73fa-4232-b72d-622f1d96d749) Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:15:47.196410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:15:47.198067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:15:47.198067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:15:47.198067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:15:47.198067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:15:47.198067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:15:47.198067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:15:47.198067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:15:47.198067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:15:47.198067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:15:47.198067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:15:47.198067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:15:47.198067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9c24f251-73fa-4232-b72d-622f1d96d749) Sep 08 14:15:47.198067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.198067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:15:47.198067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.198067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.198067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 08 14:15:47.201233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 08 14:15:47.201233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 14:15:47.201233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:15:47.201233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 08 14:15:47.201233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 08 14:15:47.201233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 08 14:15:47.201233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 08 14:15:47.201233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 08 14:15:47.201233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:15:47.201233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 08 14:15:47.201233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 08 14:15:47.201233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9c24f251-73fa-4232-b72d-622f1d96d749) Sep 08 14:15:47.201233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.201233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:15:47.201233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.201233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.201233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:15:47.201233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:15:47.202899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:15:47.202899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:15:47.202899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:15:47.202899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:15:47.202899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 08 14:15:47.202899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 08 14:15:47.202899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9c24f251-73fa-4232-b72d-622f1d96d749) Sep 08 14:15:47.202899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.202899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:15:47.202899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.202899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.202899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:15:47.202899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:15:47.202899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:15:47.202899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:15:47.202899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9c24f251-73fa-4232-b72d-622f1d96d749) Sep 08 14:15:47.202899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.204567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-54b9d868-f8e8-4d3c-be82-c7395c700551 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-9c24f251-73fa-4232-b72d-622f1d96d749) {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:15:47.204567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-54b9d868-f8e8-4d3c-be82-c7395c700551 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70" status: 400 len: 269 microversion: 2.1 time: 135.348499 Sep 08 14:15:47.204567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:15:47 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:15:47.204567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:15:47 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (10.222.0.25) Sep 08 14:15:47.204567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-54b9d868-f8e8-4d3c-be82-c7395c700551 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Unhandled error: OSError: write error Sep 08 14:15:47.204567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:15:47.204567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:15:47.205298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1357/2412] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:13:31 2022] DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 => generated 0 bytes in 135350 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:15:47.205298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-de7926d6-b97f-4511-b1d9-2bd52c8e448b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:15:47.205298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-de7926d6-b97f-4511-b1d9-2bd52c8e448b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:15:47.207967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de7926d6-b97f-4511-b1d9-2bd52c8e448b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.208459 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de7926d6-b97f-4511-b1d9-2bd52c8e448b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.208911 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de7926d6-b97f-4511-b1d9-2bd52c8e448b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.209610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de7926d6-b97f-4511-b1d9-2bd52c8e448b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.210067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de7926d6-b97f-4511-b1d9-2bd52c8e448b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.225784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de7926d6-b97f-4511-b1d9-2bd52c8e448b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.228406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-63908f62-e5e5-4d45-a7de-c35ba557d897 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0aebc7d7-85e2-490f-b733-0c480ac21ed8) Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-63908f62-e5e5-4d45-a7de-c35ba557d897 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Request method failure captured: Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 HTTP/1.1 Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfSNyJo8zI7plzmoT3joT1LB7boaKVSoM2Btn54mkWpdqceI6SLQFx4_Rpb5l-mqzqX661NrDxpQinv-CjzSmbNBVW650b02MoLz_tOe3RLJyylDgcU4AMsS5LTkC_9jOXC-JdkvqUTGjOl36FG-vYox8h_LoJX8tIcUOcoYc5HdH0s Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:15:47.232020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:15:47.233486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:15:47.233486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:15:47.233486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:15:47.233486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:15:47.233486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:15:47.233486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:15:47.233486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:15:47.233486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 5ae9377cf36e4086aa38c07b021fdead Sep 08 14:15:47.233486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:15:47.233486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:15:47.234240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0aebc7d7-85e2-490f-b733-0c480ac21ed8) Sep 08 14:15:47.234240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'server_id': '7ec70561-b436-4d03-bd60-6c5ae232b492', 'id': 'fa5de96f-82e6-44dd-a208-403b6087dd70'} Sep 08 14:15:47.234240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:15:47.234240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.234240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:15:47.234240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:15:47.234240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:15:47.234240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:15:47.234240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:15:47.234240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:15:47.234240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:15:47.234240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:15:47.234240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:15:47.234240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:15:47.234240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:15:47.234240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:15:47.234240 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0aebc7d7-85e2-490f-b733-0c480ac21ed8) Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:15:47.235406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:15:47.236746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:15:47.236746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:15:47.236746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:15:47.236746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:15:47.236746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:15:47.236746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:15:47.236746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:15:47.236746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:15:47.236746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:15:47.236746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:15:47.236746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:15:47.236746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0aebc7d7-85e2-490f-b733-0c480ac21ed8) Sep 08 14:15:47.236746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.236746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:15:47.236746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.236746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.236746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 08 14:15:47.237900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 08 14:15:47.237900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 14:15:47.237900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:15:47.237900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 08 14:15:47.237900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 08 14:15:47.237900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 08 14:15:47.237900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 08 14:15:47.237900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 08 14:15:47.237900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:15:47.237900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 08 14:15:47.237900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 08 14:15:47.237900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0aebc7d7-85e2-490f-b733-0c480ac21ed8) Sep 08 14:15:47.237900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.237900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:15:47.237900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.237900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.237900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:15:47.237900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:15:47.240278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:15:47.240278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:15:47.240278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:15:47.240278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:15:47.240278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 08 14:15:47.240278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 08 14:15:47.240278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0aebc7d7-85e2-490f-b733-0c480ac21ed8) Sep 08 14:15:47.240278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.240278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:15:47.240278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.240278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.240278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:15:47.240278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:15:47.240278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:15:47.240278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:15:47.240278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0aebc7d7-85e2-490f-b733-0c480ac21ed8) Sep 08 14:15:47.240278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.243283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-63908f62-e5e5-4d45-a7de-c35ba557d897 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-0aebc7d7-85e2-490f-b733-0c480ac21ed8) {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:15:47.243283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-63908f62-e5e5-4d45-a7de-c35ba557d897 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70" status: 400 len: 269 microversion: 2.1 time: 134.765770 Sep 08 14:15:47.243283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:15:47 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:15:47.243283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:15:47 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (10.222.0.25) Sep 08 14:15:47.243283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-63908f62-e5e5-4d45-a7de-c35ba557d897 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Unhandled error: OSError: write error Sep 08 14:15:47.243283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:15:47.243283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:15:47.244063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1056/2413] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:13:32 2022] DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 => generated 0 bytes in 134769 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:15:47.255544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-72787499-4ed1-4d54-af79-d82e3c400118 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:15:47.259152 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-72787499-4ed1-4d54-af79-d82e3c400118 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:15:47.267729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-72787499-4ed1-4d54-af79-d82e3c400118 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.268418 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-72787499-4ed1-4d54-af79-d82e3c400118 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.269126 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-72787499-4ed1-4d54-af79-d82e3c400118 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.270152 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-72787499-4ed1-4d54-af79-d82e3c400118 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.272440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-72787499-4ed1-4d54-af79-d82e3c400118 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.273491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-72787499-4ed1-4d54-af79-d82e3c400118 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.294804 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-72787499-4ed1-4d54-af79-d82e3c400118 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] HTTP exception thrown: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-72787499-4ed1-4d54-af79-d82e3c400118 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Request method failure captured: Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 HTTP/1.1 Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfR-noSyVFO_CwfvnHm1tKMROORj39_aeYmVL57cJH5P0CKIBxGmupxXskOhl3govhUXvkZDwtrF-q1QWbqRhaHpW2fGrN3ado0HHqqQh_E0h7uwLaTdYOzkjbpdS4KTgY9PGN8tIW4uf_fu6nXCzaHGAE-HkeES6UOy8csw-kGPtKA Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:15:47.296956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:15:47.298884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:15:47.298884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-TestVolumeBootPattern-588128932 Sep 08 14:15:47.298884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:15:47.298884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:15:47.298884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:15:47.298884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:15:47.298884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:15:47.298884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:15:47.298884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:15:47.298884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:15:47.298884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:15:47.298884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '6c3f043c-0662-4b63-bbd5-3a0f68c89b94'} Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:15:47.299925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:15:47.312550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:15:47.312550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:15:47.312550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:15:47.312550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:15:47.312550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:15:47.312550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:15:47.312550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:15:47.312550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:15:47.312550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.312550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:15:47.312550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.312550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.312550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:15:47.312550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:15:47.312550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:15:47.312550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:15:47.312550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:15:47.312550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.315326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-72787499-4ed1-4d54-af79-d82e3c400118 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Returning 404 to user: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:15:47.315326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-72787499-4ed1-4d54-af79-d82e3c400118 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 404 len: 111 microversion: 2.1 time: 0.066347 Sep 08 14:15:47.315326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:15:47 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 (ip 10.222.0.25) !!! Sep 08 14:15:47.315326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:15:47 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 (10.222.0.25) Sep 08 14:15:47.315326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-72787499-4ed1-4d54-af79-d82e3c400118 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Unhandled error: OSError: write error Sep 08 14:15:47.315326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:15:47.315326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:15:47.315326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1057/2414] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:15:47 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 0 bytes in 68 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:15:47.359904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-de7926d6-b97f-4511-b1d9-2bd52c8e448b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:15:47.367982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-de7926d6-b97f-4511-b1d9-2bd52c8e448b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.172968 Sep 08 14:15:47.368796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:15:47 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (ip 10.222.0.25) !!! Sep 08 14:15:47.368796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:15:47 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (10.222.0.25) Sep 08 14:15:47.369391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-de7926d6-b97f-4511-b1d9-2bd52c8e448b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Unhandled error: OSError: write error Sep 08 14:15:47.369391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:15:47.369391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:15:47.369679 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1358/2415] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:15:47 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 0 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:15:47.379383 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-787a4f64-0f1f-4e34-8b27-5ec6ef28e1d1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:15:47.379952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-787a4f64-0f1f-4e34-8b27-5ec6ef28e1d1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:15:47.386466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-787a4f64-0f1f-4e34-8b27-5ec6ef28e1d1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.386963 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-787a4f64-0f1f-4e34-8b27-5ec6ef28e1d1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.387400 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-787a4f64-0f1f-4e34-8b27-5ec6ef28e1d1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.388514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-787a4f64-0f1f-4e34-8b27-5ec6ef28e1d1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.388750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-787a4f64-0f1f-4e34-8b27-5ec6ef28e1d1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.389301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-787a4f64-0f1f-4e34-8b27-5ec6ef28e1d1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.404758 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-787a4f64-0f1f-4e34-8b27-5ec6ef28e1d1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-787a4f64-0f1f-4e34-8b27-5ec6ef28e1d1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 HTTP/1.1 Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:15:47.406357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:15:47.407892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:15:47.407892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:15:47.407892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:15:47.407892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:15:47.407892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:15:47.407892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:15:47.407892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:15:47.407892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:15:47.407892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:15:47.407892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:15:47.407892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:15:47.407892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '5a9f0e69-51d0-4a26-89d2-91c47d197556'} Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:15:47.408739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:15:47.410288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.411645 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-787a4f64-0f1f-4e34-8b27-5ec6ef28e1d1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:15:47.411645 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-787a4f64-0f1f-4e34-8b27-5ec6ef28e1d1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 404 len: 111 microversion: 2.1 time: 0.101861 Sep 08 14:15:47.411645 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:15:47 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 (ip 10.222.0.25) !!! Sep 08 14:15:47.411645 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:15:47 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 (10.222.0.25) Sep 08 14:15:47.411645 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-787a4f64-0f1f-4e34-8b27-5ec6ef28e1d1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Unhandled error: OSError: write error Sep 08 14:15:47.411645 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:15:47.411645 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:15:47.411645 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1058/2416] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:15:47 2022] DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 0 bytes in 103 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:15:47.413530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-341cafa6-9df3-46ba-bb00-40f4effdc211 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:15:47.414020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-341cafa6-9df3-46ba-bb00-40f4effdc211 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:15:47.420977 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-341cafa6-9df3-46ba-bb00-40f4effdc211 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.421462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-341cafa6-9df3-46ba-bb00-40f4effdc211 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.421899 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-341cafa6-9df3-46ba-bb00-40f4effdc211 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.422632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-341cafa6-9df3-46ba-bb00-40f4effdc211 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.423087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-341cafa6-9df3-46ba-bb00-40f4effdc211 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.423516 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-341cafa6-9df3-46ba-bb00-40f4effdc211 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.502358 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f30c6b29-a56a-4dbe-8353-020a1d58cfa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:15:47.504572 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f30c6b29-a56a-4dbe-8353-020a1d58cfa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:15:47.510533 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f30c6b29-a56a-4dbe-8353-020a1d58cfa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.511010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f30c6b29-a56a-4dbe-8353-020a1d58cfa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.511488 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f30c6b29-a56a-4dbe-8353-020a1d58cfa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.512164 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f30c6b29-a56a-4dbe-8353-020a1d58cfa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.512609 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f30c6b29-a56a-4dbe-8353-020a1d58cfa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.513028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f30c6b29-a56a-4dbe-8353-020a1d58cfa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.543030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f30c6b29-a56a-4dbe-8353-020a1d58cfa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.171864 Sep 08 14:15:47.543537 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:15:47 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments (ip 10.222.0.25) !!! Sep 08 14:15:47.543537 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:15:47 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments (10.222.0.25) Sep 08 14:15:47.544113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-f30c6b29-a56a-4dbe-8353-020a1d58cfa3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Unhandled error: OSError: write error Sep 08 14:15:47.544113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:15:47.544113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:15:47.544381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1359/2417] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:15:47 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 0 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:15:47.548593 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-84c77a0c-fc7f-4b75-ad8b-f18b2c61a245 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:15:47.549192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-84c77a0c-fc7f-4b75-ad8b-f18b2c61a245 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:15:47.554858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84c77a0c-fc7f-4b75-ad8b-f18b2c61a245 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.555332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84c77a0c-fc7f-4b75-ad8b-f18b2c61a245 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.555759 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84c77a0c-fc7f-4b75-ad8b-f18b2c61a245 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.556436 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84c77a0c-fc7f-4b75-ad8b-f18b2c61a245 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.556885 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84c77a0c-fc7f-4b75-ad8b-f18b2c61a245 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.557301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84c77a0c-fc7f-4b75-ad8b-f18b2c61a245 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.705708 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-84c77a0c-fc7f-4b75-ad8b-f18b2c61a245 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:15:47.713546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-84c77a0c-fc7f-4b75-ad8b-f18b2c61a245 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1569 microversion: 2.1 time: 0.168402 Sep 08 14:15:47.714104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1360/2418] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:15:47 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1569 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:15:47.718675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6988f8e9-5adc-421e-abff-2257a536ce9c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:15:47.719275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6988f8e9-5adc-421e-abff-2257a536ce9c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:15:47.725036 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6988f8e9-5adc-421e-abff-2257a536ce9c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.725522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6988f8e9-5adc-421e-abff-2257a536ce9c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.725959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6988f8e9-5adc-421e-abff-2257a536ce9c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.726671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6988f8e9-5adc-421e-abff-2257a536ce9c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.727124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6988f8e9-5adc-421e-abff-2257a536ce9c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.727551 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6988f8e9-5adc-421e-abff-2257a536ce9c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.741544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-6988f8e9-5adc-421e-abff-2257a536ce9c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] HTTP exception thrown: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6988f8e9-5adc-421e-abff-2257a536ce9c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Request method failure captured: Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 HTTP/1.1 Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfR-noSyVFO_CwfvnHm1tKMROORj39_aeYmVL57cJH5P0CKIBxGmupxXskOhl3govhUXvkZDwtrF-q1QWbqRhaHpW2fGrN3ado0HHqqQh_E0h7uwLaTdYOzkjbpdS4KTgY9PGN8tIW4uf_fu6nXCzaHGAE-HkeES6UOy8csw-kGPtKA Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:15:47.743119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:15:47.744829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:15:47.744829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-TestVolumeBootPattern-588128932 Sep 08 14:15:47.744829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:15:47.744829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:15:47.744829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:15:47.744829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:15:47.744829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:15:47.744829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:15:47.744829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:15:47.744829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:15:47.744829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:15:47.744829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': '6c3f043c-0662-4b63-bbd5-3a0f68c89b94'} Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:15:47.746596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:15:47.748418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:15:47.748418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:15:47.748418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:15:47.748418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:15:47.748418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:15:47.748418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:15:47.748418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:15:47.748418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:15:47.748418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.748418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:15:47.748418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.748418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.748418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:15:47.748418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:15:47.748418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:15:47.748418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:15:47.748418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:15:47.748418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.750116 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6988f8e9-5adc-421e-abff-2257a536ce9c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Returning 404 to user: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:15:47.750116 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6988f8e9-5adc-421e-abff-2257a536ce9c tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 404 len: 111 microversion: 2.1 time: 0.029344 Sep 08 14:15:47.750116 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1361/2419] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:15:47 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:15:47.750434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e538a7f8-6285-49e2-93df-3e668e48ab7e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:15:47.750787 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e538a7f8-6285-49e2-93df-3e668e48ab7e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:15:47.756494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e538a7f8-6285-49e2-93df-3e668e48ab7e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.756972 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e538a7f8-6285-49e2-93df-3e668e48ab7e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.757407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e538a7f8-6285-49e2-93df-3e668e48ab7e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.758125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e538a7f8-6285-49e2-93df-3e668e48ab7e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.758599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e538a7f8-6285-49e2-93df-3e668e48ab7e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.759024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e538a7f8-6285-49e2-93df-3e668e48ab7e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.775228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-e538a7f8-6285-49e2-93df-3e668e48ab7e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e538a7f8-6285-49e2-93df-3e668e48ab7e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 HTTP/1.1 Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:15:47.776867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:15:47.778620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:15:47.778620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:15:47.778620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:15:47.778620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:15:47.778620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:15:47.778620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:15:47.778620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:15:47.778620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:15:47.778620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:15:47.778620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:15:47.778620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:15:47.778620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': '5a9f0e69-51d0-4a26-89d2-91c47d197556'} Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:15:47.779699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:15:47.781423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:15:47.783364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e538a7f8-6285-49e2-93df-3e668e48ab7e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:15:47.783364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e538a7f8-6285-49e2-93df-3e668e48ab7e tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 404 len: 111 microversion: 2.1 time: 0.031790 Sep 08 14:15:47.783364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1362/2420] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:15:47 2022] DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:15:47.784647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e25fe637-725a-4c46-b71b-bd88747955a1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:15:47.786952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e25fe637-725a-4c46-b71b-bd88747955a1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:15:47.796066 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e25fe637-725a-4c46-b71b-bd88747955a1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.796662 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e25fe637-725a-4c46-b71b-bd88747955a1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.799720 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e25fe637-725a-4c46-b71b-bd88747955a1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.799720 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e25fe637-725a-4c46-b71b-bd88747955a1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.799720 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e25fe637-725a-4c46-b71b-bd88747955a1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.799720 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e25fe637-725a-4c46-b71b-bd88747955a1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.835520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e25fe637-725a-4c46-b71b-bd88747955a1 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055628 Sep 08 14:15:47.836088 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1363/2421] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:15:47 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:15:47.909917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-66ab82ce-fb77-49a0-8398-e419a84540f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:15:47.910439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-66ab82ce-fb77-49a0-8398-e419a84540f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:15:47.916462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66ab82ce-fb77-49a0-8398-e419a84540f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.916948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66ab82ce-fb77-49a0-8398-e419a84540f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.918131 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66ab82ce-fb77-49a0-8398-e419a84540f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.918843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66ab82ce-fb77-49a0-8398-e419a84540f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:47.919333 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66ab82ce-fb77-49a0-8398-e419a84540f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:47.919732 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66ab82ce-fb77-49a0-8398-e419a84540f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:47.946422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-66ab82ce-fb77-49a0-8398-e419a84540f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:15:48.282738 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-66ab82ce-fb77-49a0-8398-e419a84540f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 204 len: 0 microversion: 2.20 time: 0.445272 Sep 08 14:15:48.283415 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1364/2422] 10.222.0.25 () {60 vars in 1280 bytes} [Thu Sep 8 14:15:47 2022] DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 08 14:15:48.288733 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-07bdf074-a2b5-4218-96c2-5362fcf796ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:15:48.289393 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-07bdf074-a2b5-4218-96c2-5362fcf796ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:15:48.298067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-07bdf074-a2b5-4218-96c2-5362fcf796ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:48.298917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-07bdf074-a2b5-4218-96c2-5362fcf796ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:48.299678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-07bdf074-a2b5-4218-96c2-5362fcf796ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:48.300472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-07bdf074-a2b5-4218-96c2-5362fcf796ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:15:48.301101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-07bdf074-a2b5-4218-96c2-5362fcf796ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:15:48.301814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-07bdf074-a2b5-4218-96c2-5362fcf796ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:15:54.698922 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:15:54.699520 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:15:54 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:15:57.417896 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=6 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:57.445486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=8 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:58.293086 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=21,nova_api:UPDATE=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:58.324022 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=46,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:17:16.264067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:17:16.264716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:17:16 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:17:32.217693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-341cafa6-9df3-46ba-bb00-40f4effdc211 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lazy-loading 'services' on Instance uuid 7ec70561-b436-4d03-bd60-6c5ae232b492 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:17:32.307271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-07bdf074-a2b5-4218-96c2-5362fcf796ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lazy-loading 'services' on Instance uuid 7ec70561-b436-4d03-bd60-6c5ae232b492 {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:17:42.223622 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=4 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:17:42.311777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=4 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:18:47.560202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:18:47.561220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:18:47 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:18:50.964503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:18:50.965186 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:18:50 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:19:03.617760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:19:03.618610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:19:03 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:19:14.709523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:19:14.710077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:19:14 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:19:32.365563 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-341cafa6-9df3-46ba-bb00-40f4effdc211 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e565f6f0-04c1-455a-a4e6-cdccd06afff6) Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-341cafa6-9df3-46ba-bb00-40f4effdc211 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Request method failure captured: Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 HTTP/1.1 Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfSNyJo8zI7plzmoT3joT1LB7boaKVSoM2Btn54mkWpdqceI6SLQFx4_Rpb5l-mqzqX661NrDxpQinv-CjzSmbNBVW650b02MoLz_tOe3RLJyylDgcU4AMsS5LTkC_9jOXC-JdkvqUTGjOl36FG-vYox8h_LoJX8tIcUOcoYc5HdH0s Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:19:32.367736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:19:32.369289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:19:32.369289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:19:32.369289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:19:32.369289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:19:32.369289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:19:32.369289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:19:32.369289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:19:32.369289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 5ae9377cf36e4086aa38c07b021fdead Sep 08 14:19:32.369289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:19:32.369289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:19:32.370868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e565f6f0-04c1-455a-a4e6-cdccd06afff6) Sep 08 14:19:32.370868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'server_id': '7ec70561-b436-4d03-bd60-6c5ae232b492', 'id': 'fa5de96f-82e6-44dd-a208-403b6087dd70'} Sep 08 14:19:32.370868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:19:32.370868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:32.370868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:19:32.370868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:19:32.370868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:19:32.370868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:19:32.370868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:19:32.370868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:19:32.370868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:19:32.370868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:19:32.370868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:19:32.370868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:19:32.370868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:19:32.370868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:19:32.370868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e565f6f0-04c1-455a-a4e6-cdccd06afff6) Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:19:32.372198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:19:32.374075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:19:32.374075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:19:32.374075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:19:32.374075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:19:32.374075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:19:32.374075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:19:32.374075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:19:32.374075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:19:32.374075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:19:32.374075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:19:32.374075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:19:32.374075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e565f6f0-04c1-455a-a4e6-cdccd06afff6) Sep 08 14:19:32.374075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.374075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:19:32.374075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.374075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:32.374075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 08 14:19:32.377219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 08 14:19:32.377219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 14:19:32.377219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:19:32.377219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 08 14:19:32.377219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 08 14:19:32.377219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 08 14:19:32.377219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 08 14:19:32.377219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 08 14:19:32.377219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:19:32.377219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 08 14:19:32.377219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 08 14:19:32.377219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e565f6f0-04c1-455a-a4e6-cdccd06afff6) Sep 08 14:19:32.377219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.377219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:19:32.377219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.377219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:32.377219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:19:32.377219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:19:32.378746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:19:32.378746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:19:32.378746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:19:32.378746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:19:32.378746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 08 14:19:32.378746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 08 14:19:32.378746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e565f6f0-04c1-455a-a4e6-cdccd06afff6) Sep 08 14:19:32.378746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.378746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:19:32.378746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.378746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:32.378746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:19:32.378746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:19:32.378746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:19:32.378746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:19:32.378746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e565f6f0-04c1-455a-a4e6-cdccd06afff6) Sep 08 14:19:32.378746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.380702 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-341cafa6-9df3-46ba-bb00-40f4effdc211 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e565f6f0-04c1-455a-a4e6-cdccd06afff6) {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:19:32.380702 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-341cafa6-9df3-46ba-bb00-40f4effdc211 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70" status: 400 len: 269 microversion: 2.1 time: 224.959399 Sep 08 14:19:32.380702 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:19:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:19:32.380702 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:19:32 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (10.222.0.25) Sep 08 14:19:32.380702 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-341cafa6-9df3-46ba-bb00-40f4effdc211 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Unhandled error: OSError: write error Sep 08 14:19:32.380702 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:19:32.380702 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:19:32.381449 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1059/2423] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:15:47 2022] DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 => generated 0 bytes in 224961 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:19:32.413823 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-07bdf074-a2b5-4218-96c2-5362fcf796ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-37c8bd4a-f2d9-48bf-bd79-591b1e8f1f75) Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-07bdf074-a2b5-4218-96c2-5362fcf796ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Request method failure captured: Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 HTTP/1.1 Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfSNyJo8zI7plzmoT3joT1LB7boaKVSoM2Btn54mkWpdqceI6SLQFx4_Rpb5l-mqzqX661NrDxpQinv-CjzSmbNBVW650b02MoLz_tOe3RLJyylDgcU4AMsS5LTkC_9jOXC-JdkvqUTGjOl36FG-vYox8h_LoJX8tIcUOcoYc5HdH0s Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:19:32.416133 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:19:32.418592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:19:32.418592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:19:32.418592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:19:32.418592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:19:32.418592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:19:32.418592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:19:32.418592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:19:32.418592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 5ae9377cf36e4086aa38c07b021fdead Sep 08 14:19:32.418592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:19:32.418592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:19:32.419560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-37c8bd4a-f2d9-48bf-bd79-591b1e8f1f75) Sep 08 14:19:32.419560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'server_id': '7ec70561-b436-4d03-bd60-6c5ae232b492', 'id': 'fa5de96f-82e6-44dd-a208-403b6087dd70'} Sep 08 14:19:32.419560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:19:32.419560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:32.419560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:19:32.419560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:19:32.419560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:19:32.419560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:19:32.419560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:19:32.419560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:19:32.419560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:19:32.419560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:19:32.419560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:19:32.419560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:19:32.419560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:19:32.419560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:19:32.419560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-37c8bd4a-f2d9-48bf-bd79-591b1e8f1f75) Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:19:32.421564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:19:32.423793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:19:32.423793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:19:32.423793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:19:32.423793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:19:32.423793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:19:32.423793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:19:32.423793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:19:32.423793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:19:32.423793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:19:32.423793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:19:32.423793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:19:32.423793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-37c8bd4a-f2d9-48bf-bd79-591b1e8f1f75) Sep 08 14:19:32.423793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.423793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:19:32.423793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.423793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:32.423793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 08 14:19:32.427012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 08 14:19:32.427012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 14:19:32.427012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:19:32.427012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 08 14:19:32.427012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 08 14:19:32.427012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 08 14:19:32.427012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 08 14:19:32.427012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 08 14:19:32.427012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:19:32.427012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 08 14:19:32.427012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 08 14:19:32.427012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-37c8bd4a-f2d9-48bf-bd79-591b1e8f1f75) Sep 08 14:19:32.427012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.427012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:19:32.427012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.427012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:32.427012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:19:32.427012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:19:32.432127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:19:32.432127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:19:32.432127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:19:32.432127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:19:32.432127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 08 14:19:32.432127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 08 14:19:32.432127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-37c8bd4a-f2d9-48bf-bd79-591b1e8f1f75) Sep 08 14:19:32.432127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.432127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:19:32.432127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.432127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:32.432127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:19:32.432127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:19:32.432127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:19:32.432127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:19:32.432127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-37c8bd4a-f2d9-48bf-bd79-591b1e8f1f75) Sep 08 14:19:32.432127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.435630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-07bdf074-a2b5-4218-96c2-5362fcf796ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-37c8bd4a-f2d9-48bf-bd79-591b1e8f1f75) {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:19:32.435630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-07bdf074-a2b5-4218-96c2-5362fcf796ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70" status: 400 len: 269 microversion: 2.1 time: 224.133519 Sep 08 14:19:32.435630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:19:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:19:32.435630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:19:32 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (10.222.0.25) Sep 08 14:19:32.435630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-07bdf074-a2b5-4218-96c2-5362fcf796ca tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Unhandled error: OSError: write error Sep 08 14:19:32.435630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:19:32.435630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:19:32.436584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1365/2424] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:15:48 2022] DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 => generated 0 bytes in 224136 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:19:32.452183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2eaa538b-0b9c-40d2-8aea-4d5064fdb749 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:19:32.452689 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2eaa538b-0b9c-40d2-8aea-4d5064fdb749 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:19:32.459981 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2eaa538b-0b9c-40d2-8aea-4d5064fdb749 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:32.460462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2eaa538b-0b9c-40d2-8aea-4d5064fdb749 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:32.460888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2eaa538b-0b9c-40d2-8aea-4d5064fdb749 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:32.461571 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2eaa538b-0b9c-40d2-8aea-4d5064fdb749 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:32.462163 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2eaa538b-0b9c-40d2-8aea-4d5064fdb749 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:32.463005 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2eaa538b-0b9c-40d2-8aea-4d5064fdb749 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:32.504426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2210cd4d-a0c6-4c8a-a0d1-467f4f0f9a50 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:19:32.505044 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2210cd4d-a0c6-4c8a-a0d1-467f4f0f9a50 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:19:32.519628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2210cd4d-a0c6-4c8a-a0d1-467f4f0f9a50 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:32.520136 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2210cd4d-a0c6-4c8a-a0d1-467f4f0f9a50 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:32.520737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2210cd4d-a0c6-4c8a-a0d1-467f4f0f9a50 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:32.521583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2210cd4d-a0c6-4c8a-a0d1-467f4f0f9a50 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:32.522437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2210cd4d-a0c6-4c8a-a0d1-467f4f0f9a50 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:32.522876 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2210cd4d-a0c6-4c8a-a0d1-467f4f0f9a50 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:32.537547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-2210cd4d-a0c6-4c8a-a0d1-467f4f0f9a50 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] HTTP exception thrown: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2210cd4d-a0c6-4c8a-a0d1-467f4f0f9a50 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Request method failure captured: Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 HTTP/1.1 Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfR-noSyVFO_CwfvnHm1tKMROORj39_aeYmVL57cJH5P0CKIBxGmupxXskOhl3govhUXvkZDwtrF-q1QWbqRhaHpW2fGrN3ado0HHqqQh_E0h7uwLaTdYOzkjbpdS4KTgY9PGN8tIW4uf_fu6nXCzaHGAE-HkeES6UOy8csw-kGPtKA Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:19:32.539178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:19:32.540861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:19:32.540861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-TestVolumeBootPattern-588128932 Sep 08 14:19:32.540861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:19:32.540861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:19:32.540861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:19:32.540861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:19:32.540861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:19:32.540861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:19:32.540861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:19:32.540861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:19:32.540861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:19:32.540861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': '6c3f043c-0662-4b63-bbd5-3a0f68c89b94'} Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:19:32.542693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:19:32.544447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:19:32.544447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:19:32.544447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:19:32.544447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:19:32.544447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:19:32.544447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:19:32.544447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:19:32.544447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:19:32.544447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.544447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:19:32.544447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.544447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:32.544447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:19:32.544447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:19:32.544447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:19:32.544447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:19:32.544447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:19:32.544447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.545895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2210cd4d-a0c6-4c8a-a0d1-467f4f0f9a50 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Returning 404 to user: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:19:32.545895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2210cd4d-a0c6-4c8a-a0d1-467f4f0f9a50 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 404 len: 111 microversion: 2.1 time: 0.120726 Sep 08 14:19:32.545895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:19:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 (ip 10.222.0.25) !!! Sep 08 14:19:32.545895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:19:32 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 (10.222.0.25) Sep 08 14:19:32.545895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-2210cd4d-a0c6-4c8a-a0d1-467f4f0f9a50 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Unhandled error: OSError: write error Sep 08 14:19:32.545895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:19:32.545895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:19:32.545895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1366/2425] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:19:32 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 0 bytes in 123 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:19:32.547774 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c67f09b8-893c-4cc1-93ef-7d6b135850a5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:19:32.548268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c67f09b8-893c-4cc1-93ef-7d6b135850a5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:19:32.554615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c67f09b8-893c-4cc1-93ef-7d6b135850a5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:32.555109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c67f09b8-893c-4cc1-93ef-7d6b135850a5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:32.555553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c67f09b8-893c-4cc1-93ef-7d6b135850a5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:32.556846 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c67f09b8-893c-4cc1-93ef-7d6b135850a5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:32.557029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c67f09b8-893c-4cc1-93ef-7d6b135850a5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:32.561635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c67f09b8-893c-4cc1-93ef-7d6b135850a5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:32.576470 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-c67f09b8-893c-4cc1-93ef-7d6b135850a5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c67f09b8-893c-4cc1-93ef-7d6b135850a5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 HTTP/1.1 Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:19:32.578113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:19:32.579771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:19:32.579771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:19:32.579771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:19:32.579771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:19:32.579771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:19:32.579771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:19:32.579771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:19:32.579771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:19:32.579771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:19:32.579771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:19:32.579771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:19:32.579771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': '5a9f0e69-51d0-4a26-89d2-91c47d197556'} Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:19:32.580803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:19:32.585357 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:19:32.586868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c67f09b8-893c-4cc1-93ef-7d6b135850a5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:19:32.586868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c67f09b8-893c-4cc1-93ef-7d6b135850a5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 404 len: 111 microversion: 2.1 time: 0.035515 Sep 08 14:19:32.586868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:19:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 (ip 10.222.0.25) !!! Sep 08 14:19:32.586868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:19:32 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 (10.222.0.25) Sep 08 14:19:32.586868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-c67f09b8-893c-4cc1-93ef-7d6b135850a5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Unhandled error: OSError: write error Sep 08 14:19:32.586868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:19:32.586868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:19:32.586868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1367/2426] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:19:32 2022] DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 0 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:19:32.587542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-38895847-3b6e-4693-a1a2-f6a464637598 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:19:32.594104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-38895847-3b6e-4693-a1a2-f6a464637598 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:19:32.599758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38895847-3b6e-4693-a1a2-f6a464637598 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:32.600330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38895847-3b6e-4693-a1a2-f6a464637598 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:32.600846 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38895847-3b6e-4693-a1a2-f6a464637598 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:32.601625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38895847-3b6e-4693-a1a2-f6a464637598 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:32.602178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38895847-3b6e-4693-a1a2-f6a464637598 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:32.602707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38895847-3b6e-4693-a1a2-f6a464637598 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:32.627234 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-2eaa538b-0b9c-40d2-8aea-4d5064fdb749 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:19:32.635941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2eaa538b-0b9c-40d2-8aea-4d5064fdb749 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1445 microversion: 2.1 time: 0.264026 Sep 08 14:19:32.636716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:19:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (ip 10.222.0.25) !!! Sep 08 14:19:32.636716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:19:32 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (10.222.0.25) Sep 08 14:19:32.637359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-2eaa538b-0b9c-40d2-8aea-4d5064fdb749 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Unhandled error: OSError: write error Sep 08 14:19:32.637359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:19:32.637359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:19:32.637630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1060/2427] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:19:32 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 0 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:19:32.642479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-37579c4c-09e3-43ff-ab9b-d95f155fb73a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:19:32.642994 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-37579c4c-09e3-43ff-ab9b-d95f155fb73a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:19:32.645935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-38895847-3b6e-4693-a1a2-f6a464637598 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063829 Sep 08 14:19:32.646688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:19:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments (ip 10.222.0.25) !!! Sep 08 14:19:32.647002 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:19:32 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments (10.222.0.25) Sep 08 14:19:32.647858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-38895847-3b6e-4693-a1a2-f6a464637598 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Unhandled error: OSError: write error Sep 08 14:19:32.647858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:19:32.647858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:19:32.648517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1368/2428] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:19:32 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 0 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:19:32.649035 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-37579c4c-09e3-43ff-ab9b-d95f155fb73a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:32.649539 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-37579c4c-09e3-43ff-ab9b-d95f155fb73a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:32.649968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-37579c4c-09e3-43ff-ab9b-d95f155fb73a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:32.650680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-37579c4c-09e3-43ff-ab9b-d95f155fb73a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:32.651659 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-37579c4c-09e3-43ff-ab9b-d95f155fb73a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:32.652169 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-37579c4c-09e3-43ff-ab9b-d95f155fb73a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:32.656289 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a1a0a408-d4ba-4729-80f9-075de413cdee tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:19:32.657017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a1a0a408-d4ba-4729-80f9-075de413cdee tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:19:32.670897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1a0a408-d4ba-4729-80f9-075de413cdee tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:32.671398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1a0a408-d4ba-4729-80f9-075de413cdee tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:32.671790 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1a0a408-d4ba-4729-80f9-075de413cdee tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:32.672482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1a0a408-d4ba-4729-80f9-075de413cdee tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:32.672924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1a0a408-d4ba-4729-80f9-075de413cdee tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:32.673349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a1a0a408-d4ba-4729-80f9-075de413cdee tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:32.708911 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-37579c4c-09e3-43ff-ab9b-d95f155fb73a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:19:32.841027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.compute.api [None req-37579c4c-09e3-43ff-ab9b-d95f155fb73a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Instance is already in deleting state, ignoring this request Sep 08 14:19:32.841864 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-37579c4c-09e3-43ff-ab9b-d95f155fb73a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 204 len: 0 microversion: 2.20 time: 0.203494 Sep 08 14:19:32.842425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:19:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 (ip 10.222.0.25) !!! Sep 08 14:19:32.842549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:19:32 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 (10.222.0.25) Sep 08 14:19:32.843023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-37579c4c-09e3-43ff-ab9b-d95f155fb73a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Unhandled error: OSError: write error Sep 08 14:19:32.843023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:19:32.843023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:19:32.843304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1061/2429] 10.222.0.25 () {60 vars in 1280 bytes} [Thu Sep 8 14:19:32 2022] DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:19:32.847597 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d4face7c-29b9-45ca-adc6-963121a51cb4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:19:32.848199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d4face7c-29b9-45ca-adc6-963121a51cb4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:19:32.853750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d4face7c-29b9-45ca-adc6-963121a51cb4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:32.854267 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d4face7c-29b9-45ca-adc6-963121a51cb4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:32.855346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d4face7c-29b9-45ca-adc6-963121a51cb4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:32.856035 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d4face7c-29b9-45ca-adc6-963121a51cb4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:32.856486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d4face7c-29b9-45ca-adc6-963121a51cb4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:32.856907 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d4face7c-29b9-45ca-adc6-963121a51cb4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:32.993405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d4face7c-29b9-45ca-adc6-963121a51cb4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:19:33.001956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d4face7c-29b9-45ca-adc6-963121a51cb4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1445 microversion: 2.1 time: 0.157774 Sep 08 14:19:33.002603 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:19:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (ip 10.222.0.25) !!! Sep 08 14:19:33.002603 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:19:33 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (10.222.0.25) Sep 08 14:19:33.003894 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-d4face7c-29b9-45ca-adc6-963121a51cb4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Unhandled error: OSError: write error Sep 08 14:19:33.003894 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:19:33.003894 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:19:33.004205 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1062/2430] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:19:32 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 0 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:19:33.008466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-aa1989db-8418-41c7-9a72-a16eb371612b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:19:33.009060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-aa1989db-8418-41c7-9a72-a16eb371612b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:19:33.015359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa1989db-8418-41c7-9a72-a16eb371612b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:33.015837 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa1989db-8418-41c7-9a72-a16eb371612b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:33.016276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa1989db-8418-41c7-9a72-a16eb371612b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:33.016955 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa1989db-8418-41c7-9a72-a16eb371612b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:33.017403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa1989db-8418-41c7-9a72-a16eb371612b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:33.017824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa1989db-8418-41c7-9a72-a16eb371612b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:33.032086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-aa1989db-8418-41c7-9a72-a16eb371612b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] HTTP exception thrown: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-aa1989db-8418-41c7-9a72-a16eb371612b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Request method failure captured: Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 HTTP/1.1 Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfR-noSyVFO_CwfvnHm1tKMROORj39_aeYmVL57cJH5P0CKIBxGmupxXskOhl3govhUXvkZDwtrF-q1QWbqRhaHpW2fGrN3ado0HHqqQh_E0h7uwLaTdYOzkjbpdS4KTgY9PGN8tIW4uf_fu6nXCzaHGAE-HkeES6UOy8csw-kGPtKA Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:19:33.033593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:19:33.035784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:19:33.035784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-TestVolumeBootPattern-588128932 Sep 08 14:19:33.035784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:19:33.035784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:19:33.035784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:19:33.035784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:19:33.035784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:19:33.035784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:19:33.035784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:19:33.035784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:19:33.035784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:19:33.035784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '6c3f043c-0662-4b63-bbd5-3a0f68c89b94'} Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:19:33.036628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:19:33.038021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:19:33.038021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:19:33.038021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:19:33.038021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:19:33.038021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:19:33.038021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:19:33.038021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:19:33.038021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:19:33.038021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:33.038021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:19:33.038021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:33.038021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:33.038021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:19:33.038021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:19:33.038021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:19:33.038021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:19:33.038021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:19:33.038021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:33.039410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-aa1989db-8418-41c7-9a72-a16eb371612b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Returning 404 to user: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:19:33.039410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-aa1989db-8418-41c7-9a72-a16eb371612b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 404 len: 111 microversion: 2.1 time: 0.030118 Sep 08 14:19:33.039410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:19:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 (ip 10.222.0.25) !!! Sep 08 14:19:33.039410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:19:33 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 (10.222.0.25) Sep 08 14:19:33.039410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-aa1989db-8418-41c7-9a72-a16eb371612b tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Unhandled error: OSError: write error Sep 08 14:19:33.039410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:19:33.039410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:19:33.039410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1063/2431] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:19:33 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 0 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:19:33.040422 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7a3e9357-1a8b-4ae9-a6da-6f96ce51208d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:19:33.040905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7a3e9357-1a8b-4ae9-a6da-6f96ce51208d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:19:33.046768 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a3e9357-1a8b-4ae9-a6da-6f96ce51208d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:33.047349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a3e9357-1a8b-4ae9-a6da-6f96ce51208d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:33.047782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a3e9357-1a8b-4ae9-a6da-6f96ce51208d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:33.048462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a3e9357-1a8b-4ae9-a6da-6f96ce51208d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:33.048909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a3e9357-1a8b-4ae9-a6da-6f96ce51208d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:33.049326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7a3e9357-1a8b-4ae9-a6da-6f96ce51208d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:33.062696 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-7a3e9357-1a8b-4ae9-a6da-6f96ce51208d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7a3e9357-1a8b-4ae9-a6da-6f96ce51208d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 HTTP/1.1 Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:19:33.064325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:19:33.066124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:19:33.066124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:19:33.066124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:19:33.066124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:19:33.066124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:19:33.066124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:19:33.066124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:19:33.066124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:19:33.066124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:19:33.066124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:19:33.066124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:19:33.066124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '5a9f0e69-51d0-4a26-89d2-91c47d197556'} Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:19:33.067156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:19:33.068762 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:19:33.070615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7a3e9357-1a8b-4ae9-a6da-6f96ce51208d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:19:33.070615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7a3e9357-1a8b-4ae9-a6da-6f96ce51208d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 404 len: 111 microversion: 2.1 time: 0.028714 Sep 08 14:19:33.070615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:19:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 (ip 10.222.0.25) !!! Sep 08 14:19:33.070615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:19:33 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 (10.222.0.25) Sep 08 14:19:33.070615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-7a3e9357-1a8b-4ae9-a6da-6f96ce51208d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Unhandled error: OSError: write error Sep 08 14:19:33.070615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:19:33.070615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:19:33.070615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1064/2432] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:19:33 2022] DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 0 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:19:33.071415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d9545c80-876c-4de6-8857-7f8ca91790f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:19:33.073945 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d9545c80-876c-4de6-8857-7f8ca91790f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:19:33.079750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d9545c80-876c-4de6-8857-7f8ca91790f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:33.080227 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d9545c80-876c-4de6-8857-7f8ca91790f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:33.080659 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d9545c80-876c-4de6-8857-7f8ca91790f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:33.081345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d9545c80-876c-4de6-8857-7f8ca91790f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:33.081795 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d9545c80-876c-4de6-8857-7f8ca91790f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:33.082245 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d9545c80-876c-4de6-8857-7f8ca91790f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:33.112356 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d9545c80-876c-4de6-8857-7f8ca91790f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044500 Sep 08 14:19:33.112765 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:19:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments (ip 10.222.0.25) !!! Sep 08 14:19:33.112765 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:19:33 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments (10.222.0.25) Sep 08 14:19:33.113339 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-d9545c80-876c-4de6-8857-7f8ca91790f9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Unhandled error: OSError: write error Sep 08 14:19:33.113339 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:19:33.113339 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:19:33.113622 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1065/2433] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:19:33 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 0 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:19:33.117847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c0b678a1-d6b3-47ca-b22d-513f5e17b2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:19:33.118291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c0b678a1-d6b3-47ca-b22d-513f5e17b2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:19:33.124034 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c0b678a1-d6b3-47ca-b22d-513f5e17b2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:33.124507 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c0b678a1-d6b3-47ca-b22d-513f5e17b2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:33.124995 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c0b678a1-d6b3-47ca-b22d-513f5e17b2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:33.126011 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c0b678a1-d6b3-47ca-b22d-513f5e17b2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:33.126473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c0b678a1-d6b3-47ca-b22d-513f5e17b2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:33.126822 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c0b678a1-d6b3-47ca-b22d-513f5e17b2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:33.152308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c0b678a1-d6b3-47ca-b22d-513f5e17b2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:19:33.195782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.compute.api [None req-c0b678a1-d6b3-47ca-b22d-513f5e17b2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Instance is already in deleting state, ignoring this request Sep 08 14:19:33.196597 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c0b678a1-d6b3-47ca-b22d-513f5e17b2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 204 len: 0 microversion: 2.20 time: 0.082270 Sep 08 14:19:33.197121 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:19:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 (ip 10.222.0.25) !!! Sep 08 14:19:33.197121 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:19:33 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 (10.222.0.25) Sep 08 14:19:33.197669 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-c0b678a1-d6b3-47ca-b22d-513f5e17b2fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Unhandled error: OSError: write error Sep 08 14:19:33.197669 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:19:33.197669 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:19:33.197937 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1066/2434] 10.222.0.25 () {60 vars in 1280 bytes} [Thu Sep 8 14:19:33 2022] DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:19:33.202925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2b252e33-575d-49b7-ae46-21a0f338bf41 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:19:33.203404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2b252e33-575d-49b7-ae46-21a0f338bf41 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:19:33.208837 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b252e33-575d-49b7-ae46-21a0f338bf41 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:33.209315 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b252e33-575d-49b7-ae46-21a0f338bf41 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:33.209743 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b252e33-575d-49b7-ae46-21a0f338bf41 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:33.210454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b252e33-575d-49b7-ae46-21a0f338bf41 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:19:33.210903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b252e33-575d-49b7-ae46-21a0f338bf41 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:19:33.211319 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b252e33-575d-49b7-ae46-21a0f338bf41 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:19:42.661938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=8 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:19:42.700514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=14 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:19:43.207126 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=16 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:19:43.230027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=52,nova_cell1:UPDATE=2 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:19:47.503830 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-a1a0a408-d4ba-4729-80f9-075de413cdee tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lazy-loading 'services' on Instance uuid 7ec70561-b436-4d03-bd60-6c5ae232b492 {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:19:47.621086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-2b252e33-575d-49b7-ae46-21a0f338bf41 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lazy-loading 'services' on Instance uuid 7ec70561-b436-4d03-bd60-6c5ae232b492 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:19:57.508496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=4 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:19:57.624454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=4 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:20:36.276989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:20:36.277802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:20:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:21:32.968004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-a1a0a408-d4ba-4729-80f9-075de413cdee tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b211a670-e8e7-46d4-bf52-564bb718f475) Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a1a0a408-d4ba-4729-80f9-075de413cdee tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Request method failure captured: Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 HTTP/1.1 Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfSNyJo8zI7plzmoT3joT1LB7boaKVSoM2Btn54mkWpdqceI6SLQFx4_Rpb5l-mqzqX661NrDxpQinv-CjzSmbNBVW650b02MoLz_tOe3RLJyylDgcU4AMsS5LTkC_9jOXC-JdkvqUTGjOl36FG-vYox8h_LoJX8tIcUOcoYc5HdH0s Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:21:32.970742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:21:32.974072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:21:32.974072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:21:32.974072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:21:32.974072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:21:32.974072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:21:32.974072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:21:32.974072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:21:32.974072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 5ae9377cf36e4086aa38c07b021fdead Sep 08 14:21:32.974072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:21:32.974072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:21:32.976463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b211a670-e8e7-46d4-bf52-564bb718f475) Sep 08 14:21:32.976463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'server_id': '7ec70561-b436-4d03-bd60-6c5ae232b492', 'id': 'fa5de96f-82e6-44dd-a208-403b6087dd70'} Sep 08 14:21:32.976463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:21:32.976463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:32.976463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:21:32.976463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:21:32.976463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:21:32.976463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:21:32.976463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:21:32.976463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:21:32.976463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:21:32.976463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:21:32.976463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:21:32.976463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:21:32.976463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:21:32.976463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:21:32.976463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b211a670-e8e7-46d4-bf52-564bb718f475) Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:21:32.979103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:21:32.981868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:21:32.981868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:21:32.981868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:21:32.981868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:21:32.981868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:21:32.981868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:21:32.981868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:21:32.981868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:21:32.981868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:21:32.981868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:21:32.981868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:21:32.981868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b211a670-e8e7-46d4-bf52-564bb718f475) Sep 08 14:21:32.981868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:21:32.981868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:32.981868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:21:32.981868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:32.981868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 08 14:21:32.984295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 08 14:21:32.984295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 14:21:32.984295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:21:32.984295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 08 14:21:32.984295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 08 14:21:32.984295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 08 14:21:32.984295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 08 14:21:32.984295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 08 14:21:32.984295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:21:32.984295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 08 14:21:32.984295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 08 14:21:32.984295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b211a670-e8e7-46d4-bf52-564bb718f475) Sep 08 14:21:32.984295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:21:32.984295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:32.984295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:21:32.984295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:32.984295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:21:32.984295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:21:32.987661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:21:32.987661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:21:32.987661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:21:32.987661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:21:32.987661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 08 14:21:32.987661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 08 14:21:32.987661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b211a670-e8e7-46d4-bf52-564bb718f475) Sep 08 14:21:32.987661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:21:32.987661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:32.987661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:21:32.987661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:32.987661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:21:32.987661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:21:32.987661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:21:32.987661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:21:32.987661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b211a670-e8e7-46d4-bf52-564bb718f475) Sep 08 14:21:32.987661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:21:32.990375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a1a0a408-d4ba-4729-80f9-075de413cdee tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b211a670-e8e7-46d4-bf52-564bb718f475) {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:21:32.990375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a1a0a408-d4ba-4729-80f9-075de413cdee tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70" status: 400 len: 269 microversion: 2.1 time: 120.322700 Sep 08 14:21:32.990375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:21:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:21:32.990375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:21:32 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (10.222.0.25) Sep 08 14:21:32.990375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-a1a0a408-d4ba-4729-80f9-075de413cdee tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Unhandled error: OSError: write error Sep 08 14:21:32.990375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:21:32.990375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:21:32.991527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1369/2435] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:19:32 2022] DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 => generated 0 bytes in 120326 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:21:32.991527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a8ec0080-c1f4-451f-a7f0-b4f9c8993418 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:21:32.991527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a8ec0080-c1f4-451f-a7f0-b4f9c8993418 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:21:32.994617 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a8ec0080-c1f4-451f-a7f0-b4f9c8993418 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:32.995367 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a8ec0080-c1f4-451f-a7f0-b4f9c8993418 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:32.995962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a8ec0080-c1f4-451f-a7f0-b4f9c8993418 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:32.996765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a8ec0080-c1f4-451f-a7f0-b4f9c8993418 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:32.997378 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a8ec0080-c1f4-451f-a7f0-b4f9c8993418 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:32.997994 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a8ec0080-c1f4-451f-a7f0-b4f9c8993418 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.075506 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-2b252e33-575d-49b7-ae46-21a0f338bf41 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d9affdff-4c9e-4d52-9012-ddbb9d9a822b) Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2b252e33-575d-49b7-ae46-21a0f338bf41 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Request method failure captured: Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 HTTP/1.1 Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfSNyJo8zI7plzmoT3joT1LB7boaKVSoM2Btn54mkWpdqceI6SLQFx4_Rpb5l-mqzqX661NrDxpQinv-CjzSmbNBVW650b02MoLz_tOe3RLJyylDgcU4AMsS5LTkC_9jOXC-JdkvqUTGjOl36FG-vYox8h_LoJX8tIcUOcoYc5HdH0s Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:21:33.077679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:21:33.080269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:21:33.080269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:21:33.080269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:21:33.080269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:21:33.080269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:21:33.080269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:21:33.080269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:21:33.080269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 5ae9377cf36e4086aa38c07b021fdead Sep 08 14:21:33.080269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:21:33.080269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:21:33.081021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d9affdff-4c9e-4d52-9012-ddbb9d9a822b) Sep 08 14:21:33.081021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'server_id': '7ec70561-b436-4d03-bd60-6c5ae232b492', 'id': 'fa5de96f-82e6-44dd-a208-403b6087dd70'} Sep 08 14:21:33.081021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:21:33.081021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:33.081021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:21:33.081021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:21:33.081021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:21:33.081021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:21:33.081021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:21:33.081021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:21:33.081021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:21:33.081021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:21:33.081021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:21:33.081021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:21:33.081021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:21:33.081021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:21:33.081021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d9affdff-4c9e-4d52-9012-ddbb9d9a822b) Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:21:33.082791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:21:33.084490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:21:33.084490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:21:33.084490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:21:33.084490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:21:33.084490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:21:33.084490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:21:33.084490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:21:33.084490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:21:33.084490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:21:33.084490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:21:33.084490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:21:33.084490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d9affdff-4c9e-4d52-9012-ddbb9d9a822b) Sep 08 14:21:33.084490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.084490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:33.084490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.084490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:33.084490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 08 14:21:33.088172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 08 14:21:33.088172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 14:21:33.088172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:21:33.088172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 08 14:21:33.088172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 08 14:21:33.088172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 08 14:21:33.088172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 08 14:21:33.088172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 08 14:21:33.088172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:21:33.088172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 08 14:21:33.088172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 08 14:21:33.088172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d9affdff-4c9e-4d52-9012-ddbb9d9a822b) Sep 08 14:21:33.088172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.088172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:33.088172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.088172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:33.088172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:21:33.088172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:21:33.091129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:21:33.091129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:21:33.091129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:21:33.091129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:21:33.091129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 08 14:21:33.091129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 08 14:21:33.091129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d9affdff-4c9e-4d52-9012-ddbb9d9a822b) Sep 08 14:21:33.091129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.091129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:33.091129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.091129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:33.091129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:21:33.091129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:21:33.091129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:21:33.091129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:21:33.091129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d9affdff-4c9e-4d52-9012-ddbb9d9a822b) Sep 08 14:21:33.091129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.094274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2b252e33-575d-49b7-ae46-21a0f338bf41 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d9affdff-4c9e-4d52-9012-ddbb9d9a822b) {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:21:33.094274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2b252e33-575d-49b7-ae46-21a0f338bf41 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70" status: 400 len: 269 microversion: 2.1 time: 119.879879 Sep 08 14:21:33.094274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:21:33.094274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:33 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (10.222.0.25) Sep 08 14:21:33.094274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-2b252e33-575d-49b7-ae46-21a0f338bf41 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Unhandled error: OSError: write error Sep 08 14:21:33.094274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:21:33.094274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:21:33.096471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1067/2436] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:19:33 2022] DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 => generated 0 bytes in 119882 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:21:33.096471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-35ce3163-4224-47ed-b231-b4c68d2a0a55 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:21:33.096471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-35ce3163-4224-47ed-b231-b4c68d2a0a55 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:21:33.102877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-35ce3163-4224-47ed-b231-b4c68d2a0a55 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.103427 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-35ce3163-4224-47ed-b231-b4c68d2a0a55 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.103813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-35ce3163-4224-47ed-b231-b4c68d2a0a55 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.105420 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-35ce3163-4224-47ed-b231-b4c68d2a0a55 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.105420 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-35ce3163-4224-47ed-b231-b4c68d2a0a55 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.106052 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-35ce3163-4224-47ed-b231-b4c68d2a0a55 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.121346 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-35ce3163-4224-47ed-b231-b4c68d2a0a55 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] HTTP exception thrown: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-35ce3163-4224-47ed-b231-b4c68d2a0a55 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Request method failure captured: Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 HTTP/1.1 Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfR-noSyVFO_CwfvnHm1tKMROORj39_aeYmVL57cJH5P0CKIBxGmupxXskOhl3govhUXvkZDwtrF-q1QWbqRhaHpW2fGrN3ado0HHqqQh_E0h7uwLaTdYOzkjbpdS4KTgY9PGN8tIW4uf_fu6nXCzaHGAE-HkeES6UOy8csw-kGPtKA Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:21:33.127953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:21:33.135723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:21:33.135723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-TestVolumeBootPattern-588128932 Sep 08 14:21:33.135723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:21:33.135723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:21:33.135723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:21:33.135723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:21:33.135723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:21:33.135723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:21:33.135723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:21:33.135723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:21:33.135723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:21:33.135723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '6c3f043c-0662-4b63-bbd5-3a0f68c89b94'} Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:21:33.136805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:21:33.138582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:21:33.138582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:21:33.138582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:21:33.138582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:21:33.138582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:21:33.138582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:21:33.138582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:21:33.138582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:21:33.138582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.138582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:33.138582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.138582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:33.138582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:21:33.138582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:21:33.138582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:21:33.138582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:21:33.138582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:21:33.138582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.140065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-35ce3163-4224-47ed-b231-b4c68d2a0a55 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Returning 404 to user: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:21:33.140065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-35ce3163-4224-47ed-b231-b4c68d2a0a55 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 404 len: 111 microversion: 2.1 time: 0.047913 Sep 08 14:21:33.140065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 (ip 10.222.0.25) !!! Sep 08 14:21:33.140065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:33 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 (10.222.0.25) Sep 08 14:21:33.140065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-35ce3163-4224-47ed-b231-b4c68d2a0a55 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Unhandled error: OSError: write error Sep 08 14:21:33.140065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:21:33.140065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:21:33.140065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1068/2437] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:21:33 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 0 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:21:33.158692 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a8ec0080-c1f4-451f-a7f0-b4f9c8993418 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:21:33.166887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a8ec0080-c1f4-451f-a7f0-b4f9c8993418 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1445 microversion: 2.1 time: 0.190244 Sep 08 14:21:33.167414 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:21:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (ip 10.222.0.25) !!! Sep 08 14:21:33.167414 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:21:33 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (10.222.0.25) Sep 08 14:21:33.167991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-a8ec0080-c1f4-451f-a7f0-b4f9c8993418 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Unhandled error: OSError: write error Sep 08 14:21:33.167991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:21:33.167991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:21:33.168448 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1370/2438] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:21:32 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 0 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:21:33.258858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cffcf25c-7d60-462c-9965-9b3ff7d45eea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:21:33.262442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cffcf25c-7d60-462c-9965-9b3ff7d45eea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:21:33.270002 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cffcf25c-7d60-462c-9965-9b3ff7d45eea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.270526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cffcf25c-7d60-462c-9965-9b3ff7d45eea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.276034 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cffcf25c-7d60-462c-9965-9b3ff7d45eea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.276034 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cffcf25c-7d60-462c-9965-9b3ff7d45eea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.276034 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cffcf25c-7d60-462c-9965-9b3ff7d45eea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.276034 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cffcf25c-7d60-462c-9965-9b3ff7d45eea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.280799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bf7939e1-2884-4d38-aaf7-aa00e47a06c8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:21:33.281293 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bf7939e1-2884-4d38-aaf7-aa00e47a06c8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:21:33.287379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf7939e1-2884-4d38-aaf7-aa00e47a06c8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf7939e1-2884-4d38-aaf7-aa00e47a06c8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.288268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf7939e1-2884-4d38-aaf7-aa00e47a06c8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.288946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf7939e1-2884-4d38-aaf7-aa00e47a06c8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.289594 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf7939e1-2884-4d38-aaf7-aa00e47a06c8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.289811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bf7939e1-2884-4d38-aaf7-aa00e47a06c8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.313172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-bf7939e1-2884-4d38-aaf7-aa00e47a06c8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:33.314806 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cffcf25c-7d60-462c-9965-9b3ff7d45eea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.145305 Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bf7939e1-2884-4d38-aaf7-aa00e47a06c8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 HTTP/1.1 Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:21:33.315098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:21:33.316880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:21:33.316880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:21:33.316880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:21:33.316880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:21:33.316880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:21:33.316880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:21:33.316880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:21:33.316880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:21:33.316880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:21:33.316880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:21:33.316880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:33.316880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '5a9f0e69-51d0-4a26-89d2-91c47d197556'} Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:21:33.317897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:33.319374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.321349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bf7939e1-2884-4d38-aaf7-aa00e47a06c8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:21:33.321349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bf7939e1-2884-4d38-aaf7-aa00e47a06c8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 404 len: 111 microversion: 2.1 time: 0.182033 Sep 08 14:21:33.321349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 (ip 10.222.0.25) !!! Sep 08 14:21:33.321349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:33 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 (10.222.0.25) Sep 08 14:21:33.321349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-bf7939e1-2884-4d38-aaf7-aa00e47a06c8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Unhandled error: OSError: write error Sep 08 14:21:33.321349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:21:33.321349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:21:33.321349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1069/2439] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:21:33 2022] DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 0 bytes in 184 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:21:33.323350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:21:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments (ip 10.222.0.25) !!! Sep 08 14:21:33.323570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:21:33 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments (10.222.0.25) Sep 08 14:21:33.324414 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-cffcf25c-7d60-462c-9965-9b3ff7d45eea tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Unhandled error: OSError: write error Sep 08 14:21:33.324414 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:21:33.324414 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:21:33.324694 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1371/2440] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:21:33 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 0 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:21:33.330010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ed86eb4d-1fb3-4632-8ed7-74ff8a44f16a tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:21:33.330550 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ed86eb4d-1fb3-4632-8ed7-74ff8a44f16a tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:21:33.336901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed86eb4d-1fb3-4632-8ed7-74ff8a44f16a tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.337401 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed86eb4d-1fb3-4632-8ed7-74ff8a44f16a tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.338698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed86eb4d-1fb3-4632-8ed7-74ff8a44f16a tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.340317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed86eb4d-1fb3-4632-8ed7-74ff8a44f16a tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.340931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed86eb4d-1fb3-4632-8ed7-74ff8a44f16a tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.341514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed86eb4d-1fb3-4632-8ed7-74ff8a44f16a tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.458309 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e03baabb-a9ec-46eb-9c9f-121a9e52bf16 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:21:33.458940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e03baabb-a9ec-46eb-9c9f-121a9e52bf16 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:21:33.465094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e03baabb-a9ec-46eb-9c9f-121a9e52bf16 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.465586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e03baabb-a9ec-46eb-9c9f-121a9e52bf16 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.466188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e03baabb-a9ec-46eb-9c9f-121a9e52bf16 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.466873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e03baabb-a9ec-46eb-9c9f-121a9e52bf16 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.467329 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e03baabb-a9ec-46eb-9c9f-121a9e52bf16 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.467834 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e03baabb-a9ec-46eb-9c9f-121a9e52bf16 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.503642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e03baabb-a9ec-46eb-9c9f-121a9e52bf16 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:21:33.636781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.compute.api [None req-e03baabb-a9ec-46eb-9c9f-121a9e52bf16 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Instance is already in deleting state, ignoring this request Sep 08 14:21:33.637682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e03baabb-a9ec-46eb-9c9f-121a9e52bf16 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 204 len: 0 microversion: 2.20 time: 0.317510 Sep 08 14:21:33.638525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 (ip 10.222.0.25) !!! Sep 08 14:21:33.638525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:33 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 (10.222.0.25) Sep 08 14:21:33.639861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-e03baabb-a9ec-46eb-9c9f-121a9e52bf16 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Unhandled error: OSError: write error Sep 08 14:21:33.639861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:21:33.639861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:21:33.639861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1070/2441] 10.222.0.25 () {60 vars in 1280 bytes} [Thu Sep 8 14:21:33 2022] DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:21:33.644150 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-af82d591-0921-4562-9f87-2fe0e88c0496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:21:33.644751 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-af82d591-0921-4562-9f87-2fe0e88c0496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:21:33.650985 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-af82d591-0921-4562-9f87-2fe0e88c0496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.651464 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-af82d591-0921-4562-9f87-2fe0e88c0496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.651884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-af82d591-0921-4562-9f87-2fe0e88c0496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.652570 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-af82d591-0921-4562-9f87-2fe0e88c0496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.653062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-af82d591-0921-4562-9f87-2fe0e88c0496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.653480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-af82d591-0921-4562-9f87-2fe0e88c0496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.788796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-af82d591-0921-4562-9f87-2fe0e88c0496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:21:33.797071 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-af82d591-0921-4562-9f87-2fe0e88c0496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1445 microversion: 2.1 time: 0.156616 Sep 08 14:21:33.797576 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (ip 10.222.0.25) !!! Sep 08 14:21:33.797576 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:33 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (10.222.0.25) Sep 08 14:21:33.798156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-af82d591-0921-4562-9f87-2fe0e88c0496 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Unhandled error: OSError: write error Sep 08 14:21:33.798156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:21:33.798156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:21:33.798445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1071/2442] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:21:33 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 0 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:21:33.802539 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-304f67ad-c808-4702-b9aa-0daa43387da2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:21:33.803148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-304f67ad-c808-4702-b9aa-0daa43387da2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:21:33.808743 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-304f67ad-c808-4702-b9aa-0daa43387da2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.809215 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-304f67ad-c808-4702-b9aa-0daa43387da2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.809649 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-304f67ad-c808-4702-b9aa-0daa43387da2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.810775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-304f67ad-c808-4702-b9aa-0daa43387da2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.811222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-304f67ad-c808-4702-b9aa-0daa43387da2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.811636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-304f67ad-c808-4702-b9aa-0daa43387da2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.825006 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-304f67ad-c808-4702-b9aa-0daa43387da2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] HTTP exception thrown: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-304f67ad-c808-4702-b9aa-0daa43387da2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Request method failure captured: Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 HTTP/1.1 Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfR-noSyVFO_CwfvnHm1tKMROORj39_aeYmVL57cJH5P0CKIBxGmupxXskOhl3govhUXvkZDwtrF-q1QWbqRhaHpW2fGrN3ado0HHqqQh_E0h7uwLaTdYOzkjbpdS4KTgY9PGN8tIW4uf_fu6nXCzaHGAE-HkeES6UOy8csw-kGPtKA Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:21:33.826587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:21:33.828349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:21:33.828349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-TestVolumeBootPattern-588128932 Sep 08 14:21:33.828349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:21:33.828349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:21:33.828349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:21:33.828349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:21:33.828349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:21:33.828349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:21:33.828349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:21:33.828349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:21:33.828349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:21:33.828349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '6c3f043c-0662-4b63-bbd5-3a0f68c89b94'} Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:21:33.829345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:21:33.831083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:21:33.831083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:21:33.831083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:21:33.831083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:21:33.831083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:21:33.831083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:21:33.831083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:21:33.831083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:21:33.831083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.831083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:33.831083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.831083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:33.831083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:21:33.831083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:21:33.831083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:21:33.831083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:21:33.831083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:21:33.831083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.832742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-304f67ad-c808-4702-b9aa-0daa43387da2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Returning 404 to user: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:21:33.832742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-304f67ad-c808-4702-b9aa-0daa43387da2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 404 len: 111 microversion: 2.1 time: 0.028797 Sep 08 14:21:33.832742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 (ip 10.222.0.25) !!! Sep 08 14:21:33.832742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:33 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 (10.222.0.25) Sep 08 14:21:33.832742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-304f67ad-c808-4702-b9aa-0daa43387da2 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Unhandled error: OSError: write error Sep 08 14:21:33.832742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:21:33.832742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:21:33.832742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1072/2443] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:21:33 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 0 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:21:33.833433 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a1e27da6-e98f-4408-8e83-65fd70b06688 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:21:33.833810 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a1e27da6-e98f-4408-8e83-65fd70b06688 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:21:33.839998 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1e27da6-e98f-4408-8e83-65fd70b06688 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.840465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1e27da6-e98f-4408-8e83-65fd70b06688 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.840886 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1e27da6-e98f-4408-8e83-65fd70b06688 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.841557 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1e27da6-e98f-4408-8e83-65fd70b06688 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.842077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1e27da6-e98f-4408-8e83-65fd70b06688 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.842949 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1e27da6-e98f-4408-8e83-65fd70b06688 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.857041 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-a1e27da6-e98f-4408-8e83-65fd70b06688 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a1e27da6-e98f-4408-8e83-65fd70b06688 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 HTTP/1.1 Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:21:33.858577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:21:33.860292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:21:33.860292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:21:33.860292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:21:33.860292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:21:33.860292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:21:33.860292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:21:33.860292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:21:33.860292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:21:33.860292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:21:33.860292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:21:33.860292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:33.860292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '5a9f0e69-51d0-4a26-89d2-91c47d197556'} Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:21:33.861272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:33.862947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:33.864569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a1e27da6-e98f-4408-8e83-65fd70b06688 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:21:33.864569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a1e27da6-e98f-4408-8e83-65fd70b06688 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 404 len: 111 microversion: 2.1 time: 0.030020 Sep 08 14:21:33.864569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 (ip 10.222.0.25) !!! Sep 08 14:21:33.864569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:33 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 (10.222.0.25) Sep 08 14:21:33.864569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-a1e27da6-e98f-4408-8e83-65fd70b06688 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Unhandled error: OSError: write error Sep 08 14:21:33.864569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:21:33.864569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:21:33.864569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1073/2444] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:21:33 2022] DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 0 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:21:33.865539 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-06448399-bee3-4ca0-9d7d-0b63f04e7ddc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:21:33.868062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-06448399-bee3-4ca0-9d7d-0b63f04e7ddc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:21:33.873880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-06448399-bee3-4ca0-9d7d-0b63f04e7ddc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.874387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-06448399-bee3-4ca0-9d7d-0b63f04e7ddc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.874912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-06448399-bee3-4ca0-9d7d-0b63f04e7ddc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.875684 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-06448399-bee3-4ca0-9d7d-0b63f04e7ddc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.876143 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-06448399-bee3-4ca0-9d7d-0b63f04e7ddc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.876821 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-06448399-bee3-4ca0-9d7d-0b63f04e7ddc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.906922 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-06448399-bee3-4ca0-9d7d-0b63f04e7ddc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044947 Sep 08 14:21:33.907465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments (ip 10.222.0.25) !!! Sep 08 14:21:33.907465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:33 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments (10.222.0.25) Sep 08 14:21:33.908195 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-06448399-bee3-4ca0-9d7d-0b63f04e7ddc tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Unhandled error: OSError: write error Sep 08 14:21:33.908195 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:21:33.908195 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:21:33.908503 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1074/2445] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:21:33 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 0 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:21:33.913290 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-17a0f74b-c4e4-4b08-b7e2-18eb33748724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:21:33.913766 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-17a0f74b-c4e4-4b08-b7e2-18eb33748724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:21:33.921083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-17a0f74b-c4e4-4b08-b7e2-18eb33748724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.921561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-17a0f74b-c4e4-4b08-b7e2-18eb33748724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.921986 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-17a0f74b-c4e4-4b08-b7e2-18eb33748724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.922688 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-17a0f74b-c4e4-4b08-b7e2-18eb33748724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:33.923134 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-17a0f74b-c4e4-4b08-b7e2-18eb33748724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:33.923550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-17a0f74b-c4e4-4b08-b7e2-18eb33748724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:33.948552 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-17a0f74b-c4e4-4b08-b7e2-18eb33748724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:21:34.000468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.compute.api [None req-17a0f74b-c4e4-4b08-b7e2-18eb33748724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Instance is already in deleting state, ignoring this request Sep 08 14:21:34.001268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-17a0f74b-c4e4-4b08-b7e2-18eb33748724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 204 len: 0 microversion: 2.20 time: 0.091977 Sep 08 14:21:34.002035 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 (ip 10.222.0.25) !!! Sep 08 14:21:34.002035 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:34 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 (10.222.0.25) Sep 08 14:21:34.002625 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-17a0f74b-c4e4-4b08-b7e2-18eb33748724 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Unhandled error: OSError: write error Sep 08 14:21:34.002625 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:21:34.002625 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:21:34.002911 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1075/2446] 10.222.0.25 () {60 vars in 1280 bytes} [Thu Sep 8 14:21:33 2022] DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:21:34.056334 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-e657dc39-a462-4541-894b-9ea63e924158 req-75d17820-2f1c-4c2e-bbdd-2def16ebe25b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c14f6817-e3ab-4a36-ba23-590498bf1b25", "name": "network-vif-deleted", "tag": "4894f431-b6f6-46b3-95db-e9eb88227e33"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:21:34.065479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-e657dc39-a462-4541-894b-9ea63e924158 req-75d17820-2f1c-4c2e-bbdd-2def16ebe25b service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:34.065948 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-e657dc39-a462-4541-894b-9ea63e924158 req-75d17820-2f1c-4c2e-bbdd-2def16ebe25b service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:34.066405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-e657dc39-a462-4541-894b-9ea63e924158 req-75d17820-2f1c-4c2e-bbdd-2def16ebe25b service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:34.077742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-e657dc39-a462-4541-894b-9ea63e924158 req-75d17820-2f1c-4c2e-bbdd-2def16ebe25b service nova] Creating event network-vif-deleted:4894f431-b6f6-46b3-95db-e9eb88227e33 for instance c14f6817-e3ab-4a36-ba23-590498bf1b25 on hv-cinder-85646 Sep 08 14:21:34.085138 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-e657dc39-a462-4541-894b-9ea63e924158 req-75d17820-2f1c-4c2e-bbdd-2def16ebe25b service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.080928 Sep 08 14:21:34.088446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.25) !!! Sep 08 14:21:34.088446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:21:34 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /compute/v2.1/os-server-external-events (10.222.0.25) Sep 08 14:21:34.088446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [req-e657dc39-a462-4541-894b-9ea63e924158 req-75d17820-2f1c-4c2e-bbdd-2def16ebe25b service nova] Unhandled error: OSError: write error Sep 08 14:21:34.088446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:21:34.088446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:21:34.088446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1076/2447] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:21:34 2022] POST /compute/v2.1/os-server-external-events => generated 0 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:21:34.091844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0c8ad66d-83a8-40d8-908c-6f93a311959b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:21:34.092521 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0c8ad66d-83a8-40d8-908c-6f93a311959b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:21:34.098543 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0c8ad66d-83a8-40d8-908c-6f93a311959b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:34.099015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0c8ad66d-83a8-40d8-908c-6f93a311959b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:34.099483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0c8ad66d-83a8-40d8-908c-6f93a311959b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:34.100153 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0c8ad66d-83a8-40d8-908c-6f93a311959b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:34.100597 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0c8ad66d-83a8-40d8-908c-6f93a311959b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:34.101011 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0c8ad66d-83a8-40d8-908c-6f93a311959b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:34.241857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0c8ad66d-83a8-40d8-908c-6f93a311959b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:21:34.250475 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0c8ad66d-83a8-40d8-908c-6f93a311959b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1445 microversion: 2.1 time: 0.162363 Sep 08 14:21:34.251053 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1077/2448] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:21:34 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1445 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:21:34.255876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a8a1844d-a8fa-499f-b9ae-b8e7c29b60a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:21:34.256468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a8a1844d-a8fa-499f-b9ae-b8e7c29b60a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] [instance: 6c3f043c-0662-4b63-bbd5-3a0f68c89b94] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:21:34.269274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a8a1844d-a8fa-499f-b9ae-b8e7c29b60a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:34.269749 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a8a1844d-a8fa-499f-b9ae-b8e7c29b60a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:34.270188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a8a1844d-a8fa-499f-b9ae-b8e7c29b60a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:34.270888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a8a1844d-a8fa-499f-b9ae-b8e7c29b60a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:34.271331 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a8a1844d-a8fa-499f-b9ae-b8e7c29b60a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:34.271746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a8a1844d-a8fa-499f-b9ae-b8e7c29b60a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:34.286319 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-a8a1844d-a8fa-499f-b9ae-b8e7c29b60a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] HTTP exception thrown: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a8a1844d-a8fa-499f-b9ae-b8e7c29b60a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Request method failure captured: Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 HTTP/1.1 Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfR-noSyVFO_CwfvnHm1tKMROORj39_aeYmVL57cJH5P0CKIBxGmupxXskOhl3govhUXvkZDwtrF-q1QWbqRhaHpW2fGrN3ado0HHqqQh_E0h7uwLaTdYOzkjbpdS4KTgY9PGN8tIW4uf_fu6nXCzaHGAE-HkeES6UOy8csw-kGPtKA Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:21:34.287842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:21:34.289670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:21:34.289670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-TestVolumeBootPattern-588128932 Sep 08 14:21:34.289670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:21:34.289670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:21:34.289670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:21:34.289670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:21:34.289670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:21:34.289670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:21:34.289670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:21:34.289670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:21:34.289670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:21:34.289670 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '6c3f043c-0662-4b63-bbd5-3a0f68c89b94'} Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:21:34.290884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:21:34.292474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:21:34.292474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:21:34.292474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:21:34.292474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:21:34.292474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:21:34.292474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:21:34.292474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:21:34.292474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:21:34.292474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:34.292474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:34.292474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:34.292474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:34.292474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:21:34.292474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:21:34.292474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:21:34.292474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:21:34.292474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. Sep 08 14:21:34.292474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:34.293975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a8a1844d-a8fa-499f-b9ae-b8e7c29b60a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Returning 404 to user: Instance 6c3f043c-0662-4b63-bbd5-3a0f68c89b94 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:21:34.293975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a8a1844d-a8fa-499f-b9ae-b8e7c29b60a5 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94" status: 404 len: 111 microversion: 2.1 time: 0.036986 Sep 08 14:21:34.293975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1078/2449] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:21:34 2022] GET /compute/v2.1/servers/6c3f043c-0662-4b63-bbd5-3a0f68c89b94 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:21:34.294867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a678de01-4bed-455d-a984-7afcb6b46673 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:21:34.295351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a678de01-4bed-455d-a984-7afcb6b46673 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:21:34.301053 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a678de01-4bed-455d-a984-7afcb6b46673 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:34.301522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a678de01-4bed-455d-a984-7afcb6b46673 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:34.301945 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a678de01-4bed-455d-a984-7afcb6b46673 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:34.302665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a678de01-4bed-455d-a984-7afcb6b46673 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:34.303087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a678de01-4bed-455d-a984-7afcb6b46673 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:34.303497 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a678de01-4bed-455d-a984-7afcb6b46673 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:34.316625 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-a678de01-4bed-455d-a984-7afcb6b46673 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a678de01-4bed-455d-a984-7afcb6b46673 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 HTTP/1.1 Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:21:34.318108 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:21:34.319858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:21:34.319858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:21:34.319858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:21:34.319858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:21:34.319858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:21:34.319858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:21:34.319858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:21:34.319858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:21:34.319858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:21:34.319858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:21:34.319858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:34.319858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '5a9f0e69-51d0-4a26-89d2-91c47d197556'} Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:21:34.321214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:34.322679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:34.324086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a678de01-4bed-455d-a984-7afcb6b46673 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:21:34.324086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a678de01-4bed-455d-a984-7afcb6b46673 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 404 len: 111 microversion: 2.1 time: 0.028781 Sep 08 14:21:34.324086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1079/2450] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:21:34 2022] DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:21:34.325172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a45fd938-3a7a-49cc-843d-0983d59dba1a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:21:34.328371 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a45fd938-3a7a-49cc-843d-0983d59dba1a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:21:34.334378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a45fd938-3a7a-49cc-843d-0983d59dba1a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:34.334845 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a45fd938-3a7a-49cc-843d-0983d59dba1a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:34.335265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a45fd938-3a7a-49cc-843d-0983d59dba1a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:34.336491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a45fd938-3a7a-49cc-843d-0983d59dba1a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:34.336934 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a45fd938-3a7a-49cc-843d-0983d59dba1a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:34.337348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a45fd938-3a7a-49cc-843d-0983d59dba1a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:34.368409 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a45fd938-3a7a-49cc-843d-0983d59dba1a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046914 Sep 08 14:21:34.368955 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1080/2451] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:21:34 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:21:34.373320 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2b30c476-f997-474a-a19c-d6aa177025e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:21:34.373819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2b30c476-f997-474a-a19c-d6aa177025e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:21:34.379619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b30c476-f997-474a-a19c-d6aa177025e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:34.380097 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b30c476-f997-474a-a19c-d6aa177025e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:34.380524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b30c476-f997-474a-a19c-d6aa177025e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:34.381200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b30c476-f997-474a-a19c-d6aa177025e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:34.381648 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b30c476-f997-474a-a19c-d6aa177025e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:34.382064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b30c476-f997-474a-a19c-d6aa177025e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:34.410429 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2b30c476-f997-474a-a19c-d6aa177025e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:21:34.454647 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.compute.api [None req-2b30c476-f997-474a-a19c-d6aa177025e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Instance is already in deleting state, ignoring this request Sep 08 14:21:34.455448 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2b30c476-f997-474a-a19c-d6aa177025e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 204 len: 0 microversion: 2.20 time: 0.085567 Sep 08 14:21:34.456002 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1081/2452] 10.222.0.25 () {60 vars in 1280 bytes} [Thu Sep 8 14:21:34 2022] DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 08 14:21:34.460384 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4ed4f165-71e9-499b-ae3d-626741d1f683 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:21:34.461456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4ed4f165-71e9-499b-ae3d-626741d1f683 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:21:34.466924 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4ed4f165-71e9-499b-ae3d-626741d1f683 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:34.467393 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4ed4f165-71e9-499b-ae3d-626741d1f683 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:34.467849 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4ed4f165-71e9-499b-ae3d-626741d1f683 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:34.468535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4ed4f165-71e9-499b-ae3d-626741d1f683 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:34.469027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4ed4f165-71e9-499b-ae3d-626741d1f683 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:34.469443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4ed4f165-71e9-499b-ae3d-626741d1f683 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:34.482810 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-4ed4f165-71e9-499b-ae3d-626741d1f683 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4ed4f165-71e9-499b-ae3d-626741d1f683 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 HTTP/1.1 Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:21:34.484323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:21:34.487351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:21:34.487351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:21:34.487351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:21:34.487351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:21:34.487351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:21:34.487351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:21:34.487351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:21:34.487351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:21:34.487351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:21:34.487351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:21:34.487351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:34.487351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '5a9f0e69-51d0-4a26-89d2-91c47d197556'} Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:21:34.489865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:21:34.492378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:21:34.494012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4ed4f165-71e9-499b-ae3d-626741d1f683 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:21:34.494012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4ed4f165-71e9-499b-ae3d-626741d1f683 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 404 len: 111 microversion: 2.1 time: 0.028767 Sep 08 14:21:34.494012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1082/2453] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:21:34 2022] DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 111 bytes in 30 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:21:34.494012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-74e4b375-8389-4fe1-9098-a8ed6f08bdd8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:21:34.494012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-74e4b375-8389-4fe1-9098-a8ed6f08bdd8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:21:34.497204 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74e4b375-8389-4fe1-9098-a8ed6f08bdd8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:34.497684 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74e4b375-8389-4fe1-9098-a8ed6f08bdd8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:34.498109 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74e4b375-8389-4fe1-9098-a8ed6f08bdd8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:34.499001 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74e4b375-8389-4fe1-9098-a8ed6f08bdd8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:21:34.499450 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74e4b375-8389-4fe1-9098-a8ed6f08bdd8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:21:34.499860 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74e4b375-8389-4fe1-9098-a8ed6f08bdd8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:21:43.334086 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=6 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:21:43.363603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=16 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:21:44.495202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=32 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:21:44.518733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=83,nova_cell1:UPDATE=3 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:21:47.798950 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-ed86eb4d-1fb3-4632-8ed7-74ff8a44f16a tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lazy-loading 'services' on Instance uuid 7ec70561-b436-4d03-bd60-6c5ae232b492 {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:21:47.890664 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-74e4b375-8389-4fe1-9098-a8ed6f08bdd8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lazy-loading 'services' on Instance uuid d7ae43e4-11c8-417c-be73-4f9fb5c42f2c {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:21:57.811054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=4 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:21:57.896047 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=4 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:22:07.572294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:22:07.573623 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:22:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (ip 10.222.0.25) !!! Sep 08 14:22:10.976683 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:22:10.977243 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:22:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:22:23.628403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:22:23.629098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:22:23 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:23:33.832218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-ed86eb4d-1fb3-4632-8ed7-74ff8a44f16a tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-491a2f5e-4e80-4b40-acdd-afc731f3bfb2) Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ed86eb4d-1fb3-4632-8ed7-74ff8a44f16a tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Request method failure captured: Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 HTTP/1.1 Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfSNyJo8zI7plzmoT3joT1LB7boaKVSoM2Btn54mkWpdqceI6SLQFx4_Rpb5l-mqzqX661NrDxpQinv-CjzSmbNBVW650b02MoLz_tOe3RLJyylDgcU4AMsS5LTkC_9jOXC-JdkvqUTGjOl36FG-vYox8h_LoJX8tIcUOcoYc5HdH0s Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:23:33.834603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:23:33.836390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:23:33.836390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:23:33.836390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:23:33.836390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:23:33.836390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:23:33.836390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:23:33.836390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:23:33.836390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 5ae9377cf36e4086aa38c07b021fdead Sep 08 14:23:33.836390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:23:33.836390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:23:33.837426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-491a2f5e-4e80-4b40-acdd-afc731f3bfb2) Sep 08 14:23:33.837426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'server_id': '7ec70561-b436-4d03-bd60-6c5ae232b492', 'id': 'fa5de96f-82e6-44dd-a208-403b6087dd70'} Sep 08 14:23:33.837426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:23:33.837426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:23:33.837426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:23:33.837426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:23:33.837426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:23:33.837426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:23:33.837426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:23:33.837426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:23:33.837426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:23:33.837426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:23:33.837426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:23:33.837426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:23:33.837426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:23:33.837426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:23:33.837426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-491a2f5e-4e80-4b40-acdd-afc731f3bfb2) Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:23:33.839005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:23:33.842082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:23:33.842082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:23:33.842082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:23:33.842082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:23:33.842082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:23:33.842082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:23:33.842082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:23:33.842082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:23:33.842082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:23:33.842082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:23:33.842082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:23:33.842082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-491a2f5e-4e80-4b40-acdd-afc731f3bfb2) Sep 08 14:23:33.842082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:23:33.842082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:23:33.842082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:23:33.842082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:23:33.842082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 08 14:23:33.843606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 08 14:23:33.843606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 14:23:33.843606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:23:33.843606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 08 14:23:33.843606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 08 14:23:33.843606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 08 14:23:33.843606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 08 14:23:33.843606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 08 14:23:33.843606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:23:33.843606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 08 14:23:33.843606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 08 14:23:33.843606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-491a2f5e-4e80-4b40-acdd-afc731f3bfb2) Sep 08 14:23:33.843606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:23:33.843606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:23:33.843606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:23:33.843606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:23:33.843606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:23:33.843606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:23:33.845129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:23:33.845129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:23:33.845129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:23:33.845129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:23:33.845129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 08 14:23:33.845129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 08 14:23:33.845129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-491a2f5e-4e80-4b40-acdd-afc731f3bfb2) Sep 08 14:23:33.845129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:23:33.845129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:23:33.845129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:23:33.845129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:23:33.845129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:23:33.845129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:23:33.845129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:23:33.845129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:23:33.845129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-491a2f5e-4e80-4b40-acdd-afc731f3bfb2) Sep 08 14:23:33.845129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:23:33.846588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ed86eb4d-1fb3-4632-8ed7-74ff8a44f16a tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-491a2f5e-4e80-4b40-acdd-afc731f3bfb2) {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:23:33.846588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ed86eb4d-1fb3-4632-8ed7-74ff8a44f16a tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70" status: 400 len: 269 microversion: 2.1 time: 120.510525 Sep 08 14:23:33.846588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:23:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (ip 10.222.0.25) !!! Sep 08 14:23:33.846588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:23:33 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 (10.222.0.25) Sep 08 14:23:33.846588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-ed86eb4d-1fb3-4632-8ed7-74ff8a44f16a tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Unhandled error: OSError: write error Sep 08 14:23:33.846588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:23:33.846588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:23:33.847246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1372/2454] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:21:33 2022] DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492/os-volume_attachments/fa5de96f-82e6-44dd-a208-403b6087dd70 => generated 0 bytes in 120512 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:23:33.847246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d5b1aebc-e0e3-4bab-977e-5659b3767886 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:23:33.847246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d5b1aebc-e0e3-4bab-977e-5659b3767886 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:23:33.850260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d5b1aebc-e0e3-4bab-977e-5659b3767886 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:23:33.850764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d5b1aebc-e0e3-4bab-977e-5659b3767886 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:23:33.851385 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d5b1aebc-e0e3-4bab-977e-5659b3767886 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:23:33.852165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d5b1aebc-e0e3-4bab-977e-5659b3767886 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:23:33.852628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d5b1aebc-e0e3-4bab-977e-5659b3767886 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:23:33.853057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d5b1aebc-e0e3-4bab-977e-5659b3767886 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:23:33.871732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-74e4b375-8389-4fe1-9098-a8ed6f08bdd8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-af36343c-5a66-426b-9176-31ca3a77635f) Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-74e4b375-8389-4fe1-9098-a8ed6f08bdd8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Request method failure captured: Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 HTTP/1.1 Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfSsEI3OQCjaT8uW9n_C9Vd1UPOl7hGg4pp7-4nDKa37scg2VH8UGybcHZqyDn5ybTSjRS0vhV5EoGnyfNs_A0uLCWs9ZtiXuHPkrFMPvuwPxHSIVoo5s0Kpv1P1uWF7oFwsg1AH73mWHbrcQqseds7NX0AeiotGfcerp1SS6-JaE8A Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:23:33.875027 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:23:33.877627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_179292c42f8e4041adc935c80dba0f70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:23:33.877627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:23:33.877627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:23:33.877627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:23:33.877627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:23:33.877627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:23:33.877627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:23:33.877627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 1f10a95e49744a6dbb73e54e00d81add Sep 08 14:23:33.877627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:23:33.877627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:23:33.891748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-af36343c-5a66-426b-9176-31ca3a77635f) Sep 08 14:23:33.891748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'server_id': 'd7ae43e4-11c8-417c-be73-4f9fb5c42f2c', 'id': '9a4342d7-3883-4385-bcd2-b4070743be65'} Sep 08 14:23:33.891748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:23:33.891748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:23:33.891748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:23:33.891748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:23:33.891748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:23:33.891748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:23:33.891748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:23:33.891748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:23:33.891748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:23:33.891748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:23:33.891748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:23:33.891748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:23:33.891748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:23:33.891748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:23:33.891748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-af36343c-5a66-426b-9176-31ca3a77635f) Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:23:33.893263 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:23:33.896096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:23:33.896096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:23:33.896096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:23:33.896096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:23:33.896096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:23:33.896096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:23:33.896096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:23:33.896096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:23:33.896096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:23:33.896096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:23:33.896096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:23:33.896096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-af36343c-5a66-426b-9176-31ca3a77635f) Sep 08 14:23:33.896096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:23:33.896096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:23:33.896096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:23:33.896096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:23:33.896096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 08 14:23:33.897683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 08 14:23:33.897683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 14:23:33.897683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:23:33.897683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 08 14:23:33.897683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 08 14:23:33.897683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 08 14:23:33.897683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 08 14:23:33.897683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 08 14:23:33.897683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:23:33.897683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 08 14:23:33.897683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 08 14:23:33.897683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-af36343c-5a66-426b-9176-31ca3a77635f) Sep 08 14:23:33.897683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:23:33.897683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:23:33.897683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:23:33.897683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:23:33.897683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:23:33.897683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:23:33.901512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:23:33.901512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:23:33.901512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:23:33.901512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:23:33.901512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 08 14:23:33.901512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 08 14:23:33.901512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-af36343c-5a66-426b-9176-31ca3a77635f) Sep 08 14:23:33.901512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:23:33.901512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:23:33.901512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:23:33.901512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:23:33.901512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:23:33.901512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:23:33.901512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:23:33.901512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:23:33.901512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-af36343c-5a66-426b-9176-31ca3a77635f) Sep 08 14:23:33.901512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:23:33.903867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-74e4b375-8389-4fe1-9098-a8ed6f08bdd8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-af36343c-5a66-426b-9176-31ca3a77635f) {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:23:33.903867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-74e4b375-8389-4fe1-9098-a8ed6f08bdd8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65" status: 400 len: 269 microversion: 2.1 time: 119.391296 Sep 08 14:23:33.903867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:23:33 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (ip 10.222.0.25) !!! Sep 08 14:23:33.903867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:23:33 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (10.222.0.25) Sep 08 14:23:33.903867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-74e4b375-8389-4fe1-9098-a8ed6f08bdd8 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Unhandled error: OSError: write error Sep 08 14:23:33.903867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:23:33.903867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:23:33.905574 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1083/2455] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:21:34 2022] DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 => generated 0 bytes in 119396 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:23:33.906040 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d5b1aebc-e0e3-4bab-977e-5659b3767886 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:23:34.005858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-5ae28a6c-e6a9-444c-9f56-3b8fbde4eb3d req-92b8be2e-d2c6-4e31-8fcf-5ac542b900cc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "d1062abf-7525-48c5-a9cb-91da3855fd1b", "tag": "2d04f3d2-99c9-4712-bc13-6ebdffcb949f"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:23:34.031085 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-5ae28a6c-e6a9-444c-9f56-3b8fbde4eb3d req-92b8be2e-d2c6-4e31-8fcf-5ac542b900cc service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:23:34.031085 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-5ae28a6c-e6a9-444c-9f56-3b8fbde4eb3d req-92b8be2e-d2c6-4e31-8fcf-5ac542b900cc service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:23:34.031085 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-5ae28a6c-e6a9-444c-9f56-3b8fbde4eb3d req-92b8be2e-d2c6-4e31-8fcf-5ac542b900cc service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:23:34.037139 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-5ae28a6c-e6a9-444c-9f56-3b8fbde4eb3d req-92b8be2e-d2c6-4e31-8fcf-5ac542b900cc service nova] Creating event network-changed:2d04f3d2-99c9-4712-bc13-6ebdffcb949f for instance d1062abf-7525-48c5-a9cb-91da3855fd1b on hv-cinder-85646 Sep 08 14:23:34.041636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-5ae28a6c-e6a9-444c-9f56-3b8fbde4eb3d req-92b8be2e-d2c6-4e31-8fcf-5ac542b900cc service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.157355 Sep 08 14:23:34.042087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:23:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.25) !!! Sep 08 14:23:34.042087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:23:34 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /compute/v2.1/os-server-external-events (10.222.0.25) Sep 08 14:23:34.042914 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [req-5ae28a6c-e6a9-444c-9f56-3b8fbde4eb3d req-92b8be2e-d2c6-4e31-8fcf-5ac542b900cc service nova] Unhandled error: OSError: write error Sep 08 14:23:34.042914 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:23:34.042914 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:23:34.043254 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1084/2456] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:23:33 2022] POST /compute/v2.1/os-server-external-events => generated 0 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:23:34.048329 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cdb4bdab-f223-4090-b9d8-3e0621bd20fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:23:34.051458 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cdb4bdab-f223-4090-b9d8-3e0621bd20fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:23:34.055789 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cdb4bdab-f223-4090-b9d8-3e0621bd20fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:23:34.056278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cdb4bdab-f223-4090-b9d8-3e0621bd20fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:23:34.056919 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cdb4bdab-f223-4090-b9d8-3e0621bd20fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:23:34.057405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cdb4bdab-f223-4090-b9d8-3e0621bd20fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:23:34.057926 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cdb4bdab-f223-4090-b9d8-3e0621bd20fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:23:34.058379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cdb4bdab-f223-4090-b9d8-3e0621bd20fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:23:34.098447 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cdb4bdab-f223-4090-b9d8-3e0621bd20fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:23:34.232499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.compute.api [None req-cdb4bdab-f223-4090-b9d8-3e0621bd20fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Instance is already in deleting state, ignoring this request Sep 08 14:23:34.233464 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cdb4bdab-f223-4090-b9d8-3e0621bd20fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 204 len: 0 microversion: 2.20 time: 0.189141 Sep 08 14:23:34.234059 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:23:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 (ip 10.222.0.25) !!! Sep 08 14:23:34.234059 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:23:34 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 (10.222.0.25) Sep 08 14:23:34.234753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-cdb4bdab-f223-4090-b9d8-3e0621bd20fe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Unhandled error: OSError: write error Sep 08 14:23:34.234753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:23:34.234753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:23:34.235210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1085/2457] 10.222.0.25 () {60 vars in 1280 bytes} [Thu Sep 8 14:23:34 2022] DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:23:34.239484 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-482fb189-6302-4659-bc3f-9b82e9b07ff8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '.version_select at 0x7fce059a5dc0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:23:34.250519 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d5b1aebc-e0e3-4bab-977e-5659b3767886 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 204 len: 0 microversion: 2.1 time: 0.411082 Sep 08 14:23:34.253638 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:23:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (ip 10.222.0.25) !!! Sep 08 14:23:34.253873 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:23:34 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (10.222.0.25) Sep 08 14:23:34.254833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-d5b1aebc-e0e3-4bab-977e-5659b3767886 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Unhandled error: OSError: write error Sep 08 14:23:34.254833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:23:34.254833 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:23:34.255973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1373/2458] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:23:33 2022] DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 0 bytes in 418 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:23:34.261886 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-426af32d-333e-40c6-9ad3-f5fb1e3fd2e3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:23:34.262500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-426af32d-333e-40c6-9ad3-f5fb1e3fd2e3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:23:34.268647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-426af32d-333e-40c6-9ad3-f5fb1e3fd2e3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:23:34.269161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-426af32d-333e-40c6-9ad3-f5fb1e3fd2e3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:23:34.269632 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-426af32d-333e-40c6-9ad3-f5fb1e3fd2e3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:23:34.270473 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-426af32d-333e-40c6-9ad3-f5fb1e3fd2e3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:23:34.270975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-426af32d-333e-40c6-9ad3-f5fb1e3fd2e3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:23:34.271412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-426af32d-333e-40c6-9ad3-f5fb1e3fd2e3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:23:34.339357 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-482fb189-6302-4659-bc3f-9b82e9b07ff8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276" status: 202 len: 0 microversion: 2.1 time: 0.103436 Sep 08 14:23:34.339987 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1086/2459] 10.222.0.25 () {58 vars in 1218 bytes} [Thu Sep 8 14:23:34 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 => generated 0 bytes in 105 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:23:34.344813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-eb156db5-3a47-4df7-b816-e8ba0978b69c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:23:34.350110 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-eb156db5-3a47-4df7-b816-e8ba0978b69c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:23:34.352743 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb156db5-3a47-4df7-b816-e8ba0978b69c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:23:34.353221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb156db5-3a47-4df7-b816-e8ba0978b69c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:23:34.353648 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb156db5-3a47-4df7-b816-e8ba0978b69c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:23:34.354375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb156db5-3a47-4df7-b816-e8ba0978b69c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:23:34.354818 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb156db5-3a47-4df7-b816-e8ba0978b69c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:23:34.355233 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-eb156db5-3a47-4df7-b816-e8ba0978b69c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:23:34.383426 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-eb156db5-3a47-4df7-b816-e8ba0978b69c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:23:34.443272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.compute.api [None req-eb156db5-3a47-4df7-b816-e8ba0978b69c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Instance is already in deleting state, ignoring this request Sep 08 14:23:34.444099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-eb156db5-3a47-4df7-b816-e8ba0978b69c tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 204 len: 0 microversion: 2.1 time: 0.103074 Sep 08 14:23:34.444712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1087/2460] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:23:34 2022] DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:23:34.449243 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f7568ff4-23aa-4738-8b5b-e6d410013790 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:23:34.450370 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f7568ff4-23aa-4738-8b5b-e6d410013790 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:23:34.455913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f7568ff4-23aa-4738-8b5b-e6d410013790 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:23:34.456389 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f7568ff4-23aa-4738-8b5b-e6d410013790 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:23:34.456814 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f7568ff4-23aa-4738-8b5b-e6d410013790 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:23:34.457496 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f7568ff4-23aa-4738-8b5b-e6d410013790 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:23:34.457944 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f7568ff4-23aa-4738-8b5b-e6d410013790 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:23:34.458388 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f7568ff4-23aa-4738-8b5b-e6d410013790 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:23:34.485606 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f7568ff4-23aa-4738-8b5b-e6d410013790 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:23:34.675736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.compute.api [None req-f7568ff4-23aa-4738-8b5b-e6d410013790 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Instance is already in deleting state, ignoring this request Sep 08 14:23:34.677067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f7568ff4-23aa-4738-8b5b-e6d410013790 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 204 len: 0 microversion: 2.20 time: 0.231240 Sep 08 14:23:34.678048 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1088/2461] 10.222.0.25 () {60 vars in 1280 bytes} [Thu Sep 8 14:23:34 2022] DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 08 14:23:34.685682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a0c801db-2217-48b6-8d4d-dc369c32d5d9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '.version_select at 0x7fce059b11f0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:23:34.704526 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-a0c801db-2217-48b6-8d4d-dc369c32d5d9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a0c801db-2217-48b6-8d4d-dc369c32d5d9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 HTTP/1.1 Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:23:34.708221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:23:34.709825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:23:34.709825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:23:34.709825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:23:34.709825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:23:34.709825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:23:34.709825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:23:34.709825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:23:34.709825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:23:34.709825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:23:34.709825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 14:23:34.709825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:23:34.709825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'tempest-keypair-997401276'} Sep 08 14:23:34.709825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:23:34.711316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:23:34.713590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:23:34.717212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:23:34.717212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:23:34.717212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 08 14:23:34.717212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 08 14:23:34.717212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:23:34.717212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:23:34.717212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:23:34.717212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:23:34.717212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:23:34.717212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:23:34.717212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:23:34.717212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:23:34.717212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:23:34.717212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:23:34.717212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:23:34.717212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:23:34.717212 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:23:34.720187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a0c801db-2217-48b6-8d4d-dc369c32d5d9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:23:34.720187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a0c801db-2217-48b6-8d4d-dc369c32d5d9 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276" status: 404 len: 131 microversion: 2.1 time: 0.030796 Sep 08 14:23:34.720187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1089/2462] 10.222.0.25 () {58 vars in 1218 bytes} [Thu Sep 8 14:23:34 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 => generated 131 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:23:34.720187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-abc36521-6796-4d75-848c-4cc7aa09c2aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:23:34.720187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-abc36521-6796-4d75-848c-4cc7aa09c2aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:23:34.730315 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abc36521-6796-4d75-848c-4cc7aa09c2aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:23:34.730932 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abc36521-6796-4d75-848c-4cc7aa09c2aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:23:34.731436 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abc36521-6796-4d75-848c-4cc7aa09c2aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:23:34.732147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abc36521-6796-4d75-848c-4cc7aa09c2aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:23:34.732612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abc36521-6796-4d75-848c-4cc7aa09c2aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:23:34.733045 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abc36521-6796-4d75-848c-4cc7aa09c2aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:23:44.266921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=9,nova_api:UPDATE=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:23:44.303283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=18,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:23:44.725126 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=15,nova_api:DELETE=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:23:44.758994 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=45,nova_cell1:UPDATE=3 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:23:48.021453 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-426af32d-333e-40c6-9ad3-f5fb1e3fd2e3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lazy-loading 'services' on Instance uuid d7ae43e4-11c8-417c-be73-4f9fb5c42f2c {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:23:48.065231 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-abc36521-6796-4d75-848c-4cc7aa09c2aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lazy-loading 'services' on Instance uuid d7ae43e4-11c8-417c-be73-4f9fb5c42f2c {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:23:56.286560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:23:56.287394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:23:56 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (ip 10.222.0.25) !!! Sep 08 14:23:58.026577 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=4 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:23:58.069157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=4 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:25:30.985928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:25:30.986584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:25:30 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (ip 10.222.0.25) !!! Sep 08 14:25:34.152726 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-426af32d-333e-40c6-9ad3-f5fb1e3fd2e3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-776746e3-aa3d-4b91-88c7-e0d27a109f62) Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-426af32d-333e-40c6-9ad3-f5fb1e3fd2e3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Request method failure captured: Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 HTTP/1.1 Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfSsEI3OQCjaT8uW9n_C9Vd1UPOl7hGg4pp7-4nDKa37scg2VH8UGybcHZqyDn5ybTSjRS0vhV5EoGnyfNs_A0uLCWs9ZtiXuHPkrFMPvuwPxHSIVoo5s0Kpv1P1uWF7oFwsg1AH73mWHbrcQqseds7NX0AeiotGfcerp1SS6-JaE8A Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:25:34.154975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:25:34.157153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_179292c42f8e4041adc935c80dba0f70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:25:34.157153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:25:34.157153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:25:34.157153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:25:34.157153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:25:34.157153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:25:34.157153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:25:34.157153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 1f10a95e49744a6dbb73e54e00d81add Sep 08 14:25:34.157153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:25:34.157153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:25:34.158227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-776746e3-aa3d-4b91-88c7-e0d27a109f62) Sep 08 14:25:34.158227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'server_id': 'd7ae43e4-11c8-417c-be73-4f9fb5c42f2c', 'id': '9a4342d7-3883-4385-bcd2-b4070743be65'} Sep 08 14:25:34.158227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:25:34.158227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:25:34.158227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:25:34.158227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:25:34.158227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:25:34.158227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:25:34.158227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:25:34.158227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:25:34.158227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:25:34.158227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:25:34.158227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:25:34.158227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:25:34.158227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:25:34.158227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:25:34.158227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-776746e3-aa3d-4b91-88c7-e0d27a109f62) Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:25:34.160735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:25:34.165009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:25:34.165009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:25:34.165009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:25:34.165009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:25:34.165009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:25:34.165009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:25:34.165009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:25:34.165009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:25:34.165009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:25:34.165009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:25:34.165009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:25:34.165009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-776746e3-aa3d-4b91-88c7-e0d27a109f62) Sep 08 14:25:34.165009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.165009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:25:34.165009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.165009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:25:34.165009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 08 14:25:34.166546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 08 14:25:34.166546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 14:25:34.166546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:25:34.166546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 08 14:25:34.166546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 08 14:25:34.166546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 08 14:25:34.166546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 08 14:25:34.166546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 08 14:25:34.166546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:25:34.166546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 08 14:25:34.166546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 08 14:25:34.166546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-776746e3-aa3d-4b91-88c7-e0d27a109f62) Sep 08 14:25:34.166546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.166546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:25:34.166546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.166546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:25:34.166546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:25:34.166546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:25:34.168614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:25:34.168614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:25:34.168614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:25:34.168614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:25:34.168614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 08 14:25:34.168614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 08 14:25:34.168614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-776746e3-aa3d-4b91-88c7-e0d27a109f62) Sep 08 14:25:34.168614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.168614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:25:34.168614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.168614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:25:34.168614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:25:34.168614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:25:34.168614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:25:34.168614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:25:34.168614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-776746e3-aa3d-4b91-88c7-e0d27a109f62) Sep 08 14:25:34.168614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.170674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-426af32d-333e-40c6-9ad3-f5fb1e3fd2e3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-776746e3-aa3d-4b91-88c7-e0d27a109f62) {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:25:34.170674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-426af32d-333e-40c6-9ad3-f5fb1e3fd2e3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65" status: 400 len: 269 microversion: 2.1 time: 119.899187 Sep 08 14:25:34.170674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:25:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (ip 10.222.0.25) !!! Sep 08 14:25:34.170674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:25:34 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (10.222.0.25) Sep 08 14:25:34.170674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-426af32d-333e-40c6-9ad3-f5fb1e3fd2e3 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Unhandled error: OSError: write error Sep 08 14:25:34.170674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:25:34.170674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:25:34.171489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1374/2463] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:23:34 2022] DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 => generated 0 bytes in 119901 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:25:34.191373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-abc36521-6796-4d75-848c-4cc7aa09c2aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b0aecb50-c5bc-4ee4-bc1b-044123ee4b9c) Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-abc36521-6796-4d75-848c-4cc7aa09c2aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Request method failure captured: Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 HTTP/1.1 Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfSsEI3OQCjaT8uW9n_C9Vd1UPOl7hGg4pp7-4nDKa37scg2VH8UGybcHZqyDn5ybTSjRS0vhV5EoGnyfNs_A0uLCWs9ZtiXuHPkrFMPvuwPxHSIVoo5s0Kpv1P1uWF7oFwsg1AH73mWHbrcQqseds7NX0AeiotGfcerp1SS6-JaE8A Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:25:34.193807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:25:34.196523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_179292c42f8e4041adc935c80dba0f70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:25:34.196523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:25:34.196523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:25:34.196523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:25:34.196523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:25:34.196523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:25:34.196523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:25:34.196523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 1f10a95e49744a6dbb73e54e00d81add Sep 08 14:25:34.196523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:25:34.196523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:25:34.197680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b0aecb50-c5bc-4ee4-bc1b-044123ee4b9c) Sep 08 14:25:34.197680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'server_id': 'd7ae43e4-11c8-417c-be73-4f9fb5c42f2c', 'id': '9a4342d7-3883-4385-bcd2-b4070743be65'} Sep 08 14:25:34.197680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:25:34.197680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:25:34.197680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:25:34.197680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:25:34.197680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:25:34.197680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:25:34.197680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:25:34.197680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:25:34.197680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:25:34.197680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:25:34.197680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:25:34.197680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:25:34.197680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:25:34.197680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:25:34.197680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b0aecb50-c5bc-4ee4-bc1b-044123ee4b9c) Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:25:34.200895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:25:34.203583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:25:34.203583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:25:34.203583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:25:34.203583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:25:34.203583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:25:34.203583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:25:34.203583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:25:34.203583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:25:34.203583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:25:34.203583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:25:34.203583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:25:34.203583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b0aecb50-c5bc-4ee4-bc1b-044123ee4b9c) Sep 08 14:25:34.203583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.203583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:25:34.203583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.203583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:25:34.203583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 08 14:25:34.206119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 08 14:25:34.206119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 14:25:34.206119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:25:34.206119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 08 14:25:34.206119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 08 14:25:34.206119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 08 14:25:34.206119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 08 14:25:34.206119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 08 14:25:34.206119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:25:34.206119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 08 14:25:34.206119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 08 14:25:34.206119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b0aecb50-c5bc-4ee4-bc1b-044123ee4b9c) Sep 08 14:25:34.206119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.206119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:25:34.206119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.206119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:25:34.206119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:25:34.206119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:25:34.208782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:25:34.208782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:25:34.208782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:25:34.208782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:25:34.208782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 08 14:25:34.208782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 08 14:25:34.208782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b0aecb50-c5bc-4ee4-bc1b-044123ee4b9c) Sep 08 14:25:34.208782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.208782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:25:34.208782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.208782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:25:34.208782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:25:34.208782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:25:34.208782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:25:34.208782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:25:34.208782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b0aecb50-c5bc-4ee4-bc1b-044123ee4b9c) Sep 08 14:25:34.208782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.213042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-abc36521-6796-4d75-848c-4cc7aa09c2aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b0aecb50-c5bc-4ee4-bc1b-044123ee4b9c) {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:25:34.213042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-abc36521-6796-4d75-848c-4cc7aa09c2aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65" status: 400 len: 269 microversion: 2.1 time: 119.484822 Sep 08 14:25:34.213042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:25:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (ip 10.222.0.25) !!! Sep 08 14:25:34.213042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:25:34 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (10.222.0.25) Sep 08 14:25:34.213042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-abc36521-6796-4d75-848c-4cc7aa09c2aa tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Unhandled error: OSError: write error Sep 08 14:25:34.213042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:25:34.213042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:25:34.214295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1090/2464] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:23:34 2022] DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 => generated 0 bytes in 119489 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:25:34.214295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bd70962a-7aac-4ebf-925f-e35a6d412100 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:25:34.214705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bd70962a-7aac-4ebf-925f-e35a6d412100 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:25:34.222537 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd70962a-7aac-4ebf-925f-e35a6d412100 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:25:34.224057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd70962a-7aac-4ebf-925f-e35a6d412100 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:25:34.224499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd70962a-7aac-4ebf-925f-e35a6d412100 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:25:34.225208 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd70962a-7aac-4ebf-925f-e35a6d412100 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:25:34.225760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd70962a-7aac-4ebf-925f-e35a6d412100 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:25:34.226129 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd70962a-7aac-4ebf-925f-e35a6d412100 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:25:34.251540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bd70962a-7aac-4ebf-925f-e35a6d412100 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:25:34.293474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b12d1d50-f4a1-4e07-b564-37983e29117f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:25:34.293884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b12d1d50-f4a1-4e07-b564-37983e29117f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:25:34.300520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b12d1d50-f4a1-4e07-b564-37983e29117f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:25:34.303711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b12d1d50-f4a1-4e07-b564-37983e29117f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:25:34.303711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b12d1d50-f4a1-4e07-b564-37983e29117f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:25:34.303711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b12d1d50-f4a1-4e07-b564-37983e29117f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:25:34.303711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b12d1d50-f4a1-4e07-b564-37983e29117f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:25:34.304148 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b12d1d50-f4a1-4e07-b564-37983e29117f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:25:34.329196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b12d1d50-f4a1-4e07-b564-37983e29117f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:25:34.357581 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.compute.api [None req-bd70962a-7aac-4ebf-925f-e35a6d412100 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Instance is already in deleting state, ignoring this request Sep 08 14:25:34.358467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bd70962a-7aac-4ebf-925f-e35a6d412100 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 204 len: 0 microversion: 2.20 time: 0.156552 Sep 08 14:25:34.359001 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:25:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 (ip 10.222.0.25) !!! Sep 08 14:25:34.359001 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:25:34 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 (10.222.0.25) Sep 08 14:25:34.359580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-bd70962a-7aac-4ebf-925f-e35a6d412100 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Unhandled error: OSError: write error Sep 08 14:25:34.359580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:25:34.359580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:25:34.359946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1091/2465] 10.222.0.25 () {60 vars in 1280 bytes} [Thu Sep 8 14:25:34 2022] DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:25:34.364250 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d1091d6a-8323-4dfb-8cae-05124b7fc88d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '.version_select at 0x7fce05992160>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:25:34.376733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-d1091d6a-8323-4dfb-8cae-05124b7fc88d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d1091d6a-8323-4dfb-8cae-05124b7fc88d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 HTTP/1.1 Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:25:34.378561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:25:34.381391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:25:34.381391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:25:34.381391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:25:34.381391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:25:34.381391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:25:34.381391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:25:34.381391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:25:34.381391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:25:34.381391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:25:34.381391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 14:25:34.381391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:25:34.381391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'tempest-keypair-997401276'} Sep 08 14:25:34.381391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:25:34.383830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:25:34.385940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:25:34.388310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:25:34.388310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:25:34.388310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 08 14:25:34.388310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 08 14:25:34.388310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:25:34.388310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:25:34.388310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:25:34.388310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.388310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:25:34.388310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.388310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:25:34.388310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:25:34.388310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:25:34.388310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:25:34.388310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:25:34.388310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:25:34.388310 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:25:34.390197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d1091d6a-8323-4dfb-8cae-05124b7fc88d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:25:34.390197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d1091d6a-8323-4dfb-8cae-05124b7fc88d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276" status: 404 len: 131 microversion: 2.1 time: 0.019470 Sep 08 14:25:34.390197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1092/2466] 10.222.0.25 () {58 vars in 1218 bytes} [Thu Sep 8 14:25:34 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 => generated 131 bytes in 21 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:25:34.390197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cb16e6d2-aa8b-4ee6-abaa-af8de2bb9d99 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:25:34.390197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cb16e6d2-aa8b-4ee6-abaa-af8de2bb9d99 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:25:34.393946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb16e6d2-aa8b-4ee6-abaa-af8de2bb9d99 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:25:34.394472 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb16e6d2-aa8b-4ee6-abaa-af8de2bb9d99 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:25:34.394910 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb16e6d2-aa8b-4ee6-abaa-af8de2bb9d99 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:25:34.395679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb16e6d2-aa8b-4ee6-abaa-af8de2bb9d99 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:25:34.396146 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb16e6d2-aa8b-4ee6-abaa-af8de2bb9d99 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:25:34.397187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cb16e6d2-aa8b-4ee6-abaa-af8de2bb9d99 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:25:34.432602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.compute.api [None req-b12d1d50-f4a1-4e07-b564-37983e29117f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Instance is already in deleting state, ignoring this request Sep 08 14:25:34.433433 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b12d1d50-f4a1-4e07-b564-37983e29117f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 204 len: 0 microversion: 2.1 time: 0.274795 Sep 08 14:25:34.433946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:25:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (ip 10.222.0.25) !!! Sep 08 14:25:34.433946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:25:34 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (10.222.0.25) Sep 08 14:25:34.434559 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-b12d1d50-f4a1-4e07-b564-37983e29117f tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Unhandled error: OSError: write error Sep 08 14:25:34.434559 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:25:34.434559 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:25:34.434858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1375/2467] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:25:34 2022] DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 0 bytes in 277 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:25:34.439260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-58ecc74c-f977-438f-b068-5fbd17315387 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:25:34.439752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-58ecc74c-f977-438f-b068-5fbd17315387 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:25:34.446449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58ecc74c-f977-438f-b068-5fbd17315387 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:25:34.446970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58ecc74c-f977-438f-b068-5fbd17315387 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:25:34.447398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58ecc74c-f977-438f-b068-5fbd17315387 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:25:34.448095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58ecc74c-f977-438f-b068-5fbd17315387 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:25:34.448549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58ecc74c-f977-438f-b068-5fbd17315387 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:25:34.448976 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-58ecc74c-f977-438f-b068-5fbd17315387 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:25:34.474474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-58ecc74c-f977-438f-b068-5fbd17315387 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:25:34.584803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.compute.api [None req-58ecc74c-f977-438f-b068-5fbd17315387 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Instance is already in deleting state, ignoring this request Sep 08 14:25:34.585944 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-58ecc74c-f977-438f-b068-5fbd17315387 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 204 len: 0 microversion: 2.1 time: 0.150216 Sep 08 14:25:34.586785 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1376/2468] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:25:34 2022] DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:25:34.592713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4312bb73-75e4-4041-8126-a59c7351dc2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:25:34.593348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4312bb73-75e4-4041-8126-a59c7351dc2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:25:34.601012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4312bb73-75e4-4041-8126-a59c7351dc2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:25:34.601716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4312bb73-75e4-4041-8126-a59c7351dc2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:25:34.602398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4312bb73-75e4-4041-8126-a59c7351dc2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:25:34.603267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4312bb73-75e4-4041-8126-a59c7351dc2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:25:34.603867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4312bb73-75e4-4041-8126-a59c7351dc2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:25:34.604454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4312bb73-75e4-4041-8126-a59c7351dc2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:25:34.635938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4312bb73-75e4-4041-8126-a59c7351dc2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:25:34.743307 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.compute.api [None req-4312bb73-75e4-4041-8126-a59c7351dc2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Instance is already in deleting state, ignoring this request Sep 08 14:25:34.744414 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4312bb73-75e4-4041-8126-a59c7351dc2a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 204 len: 0 microversion: 2.20 time: 0.156524 Sep 08 14:25:34.745200 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1377/2469] 10.222.0.25 () {60 vars in 1280 bytes} [Thu Sep 8 14:25:34 2022] DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 08 14:25:34.751340 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4ff928c2-43d2-43a2-b839-ae93e1f85869 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:25:34.751971 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4ff928c2-43d2-43a2-b839-ae93e1f85869 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:25:34.760201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ff928c2-43d2-43a2-b839-ae93e1f85869 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:25:34.760926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ff928c2-43d2-43a2-b839-ae93e1f85869 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:25:34.761535 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ff928c2-43d2-43a2-b839-ae93e1f85869 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:25:34.762575 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ff928c2-43d2-43a2-b839-ae93e1f85869 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:25:34.763163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ff928c2-43d2-43a2-b839-ae93e1f85869 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:25:34.763753 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ff928c2-43d2-43a2-b839-ae93e1f85869 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:25:34.799413 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4ff928c2-43d2-43a2-b839-ae93e1f85869 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:25:35.117098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4ff928c2-43d2-43a2-b839-ae93e1f85869 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 204 len: 0 microversion: 2.1 time: 0.370423 Sep 08 14:25:35.117991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1378/2470] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:25:34 2022] DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:25:35.123479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ef8384f1-6bc3-4cf6-80b1-1c2eb0237368 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '.version_select at 0x7fce025513a0>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:25:35.151874 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-ef8384f1-6bc3-4cf6-80b1-1c2eb0237368 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ef8384f1-6bc3-4cf6-80b1-1c2eb0237368 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 HTTP/1.1 Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:25:35.155689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:25:35.159700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:25:35.159700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:25:35.159700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:25:35.159700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:25:35.159700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:25:35.159700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:25:35.159700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:25:35.159700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:25:35.159700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:25:35.159700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: Sep 08 14:25:35.159700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:25:35.159700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'tempest-keypair-997401276'} Sep 08 14:25:35.159700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:25:35.161512 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:25:35.164220 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:25:35.165977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:25:35.165977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:25:35.165977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 08 14:25:35.165977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 08 14:25:35.165977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:25:35.165977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:25:35.165977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:25:35.165977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:25:35.165977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:25:35.165977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:25:35.165977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:25:35.165977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:25:35.165977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:25:35.165977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:25:35.165977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:25:35.165977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:25:35.165977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:25:35.167368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ef8384f1-6bc3-4cf6-80b1-1c2eb0237368 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:25:35.167368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ef8384f1-6bc3-4cf6-80b1-1c2eb0237368 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276" status: 404 len: 131 microversion: 2.1 time: 0.037888 Sep 08 14:25:35.167368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1379/2471] 10.222.0.25 () {58 vars in 1218 bytes} [Thu Sep 8 14:25:35 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 => generated 131 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:25:35.167368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8ac31e35-822d-46c5-bb50-40eb05b76290 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:25:35.167368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8ac31e35-822d-46c5-bb50-40eb05b76290 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:25:35.173321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8ac31e35-822d-46c5-bb50-40eb05b76290 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:25:35.174223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8ac31e35-822d-46c5-bb50-40eb05b76290 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:25:35.174835 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8ac31e35-822d-46c5-bb50-40eb05b76290 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:25:35.175894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8ac31e35-822d-46c5-bb50-40eb05b76290 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:25:35.176519 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8ac31e35-822d-46c5-bb50-40eb05b76290 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:25:35.176959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8ac31e35-822d-46c5-bb50-40eb05b76290 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:25:44.391301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=6 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:25:44.418862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:25:45.168790 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=17,nova_api:UPDATE=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:25:45.196482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=57,nova_cell1:UPDATE=4,nova_cell1:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:27:16.299039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:27:16.299754 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:27:16 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (ip 10.222.0.25) !!! Sep 08 14:27:34.434755 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-cb16e6d2-aa8b-4ee6-abaa-af8de2bb9d99 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lazy-loading 'services' on Instance uuid d7ae43e4-11c8-417c-be73-4f9fb5c42f2c {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:27:34.521802 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-8ac31e35-822d-46c5-bb50-40eb05b76290 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lazy-loading 'services' on Instance uuid d7ae43e4-11c8-417c-be73-4f9fb5c42f2c {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:27:44.438953 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=4 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:27:44.526695 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=4 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:28:47.587705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:28:47.588379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:28:47 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (ip 10.222.0.25) !!! Sep 08 14:28:50.995425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:28:50.995981 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:28:50 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (ip 10.222.0.25) !!! Sep 08 14:28:55.140888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:28:55.142097 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:28:55 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (ip 10.222.0.25) !!! Sep 08 14:29:14.729905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:29:14.730587 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:29:14 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (ip 10.222.0.25) !!! Sep 08 14:29:41.080194 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-cb16e6d2-aa8b-4ee6-abaa-af8de2bb9d99 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff1ad378-b719-4704-b025-ffbee4e20edc) Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cb16e6d2-aa8b-4ee6-abaa-af8de2bb9d99 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Request method failure captured: Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 HTTP/1.1 Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfSsEI3OQCjaT8uW9n_C9Vd1UPOl7hGg4pp7-4nDKa37scg2VH8UGybcHZqyDn5ybTSjRS0vhV5EoGnyfNs_A0uLCWs9ZtiXuHPkrFMPvuwPxHSIVoo5s0Kpv1P1uWF7oFwsg1AH73mWHbrcQqseds7NX0AeiotGfcerp1SS6-JaE8A Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:29:41.083222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:29:41.084862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_179292c42f8e4041adc935c80dba0f70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:41.084862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:41.084862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:29:41.084862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:41.084862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:29:41.084862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:29:41.084862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:29:41.084862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 1f10a95e49744a6dbb73e54e00d81add Sep 08 14:29:41.084862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:29:41.084862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:29:41.085824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff1ad378-b719-4704-b025-ffbee4e20edc) Sep 08 14:29:41.085824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'server_id': 'd7ae43e4-11c8-417c-be73-4f9fb5c42f2c', 'id': '9a4342d7-3883-4385-bcd2-b4070743be65'} Sep 08 14:29:41.085824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:41.085824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.085824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:29:41.085824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:29:41.085824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:29:41.085824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:29:41.085824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:29:41.085824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:29:41.085824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:29:41.085824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:29:41.085824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:29:41.085824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:29:41.085824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:29:41.085824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:29:41.085824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff1ad378-b719-4704-b025-ffbee4e20edc) Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:29:41.087224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:29:41.088760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:29:41.088760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:29:41.088760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:29:41.088760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:29:41.088760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:29:41.088760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:29:41.088760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:29:41.088760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:29:41.088760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:29:41.088760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:29:41.088760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:29:41.088760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff1ad378-b719-4704-b025-ffbee4e20edc) Sep 08 14:29:41.088760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.088760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.088760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.088760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.088760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 08 14:29:41.092930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 08 14:29:41.092930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 14:29:41.092930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:29:41.092930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 08 14:29:41.092930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 08 14:29:41.092930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 08 14:29:41.092930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 08 14:29:41.092930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 08 14:29:41.092930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:29:41.092930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 08 14:29:41.092930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 08 14:29:41.092930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff1ad378-b719-4704-b025-ffbee4e20edc) Sep 08 14:29:41.092930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.092930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.092930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.092930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.092930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:41.092930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:41.095114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:41.095114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:41.095114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:41.095114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:41.095114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 08 14:29:41.095114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 08 14:29:41.095114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff1ad378-b719-4704-b025-ffbee4e20edc) Sep 08 14:29:41.095114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.095114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.095114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.095114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.095114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:41.095114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:41.095114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:41.095114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:41.095114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff1ad378-b719-4704-b025-ffbee4e20edc) Sep 08 14:29:41.095114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.097665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cb16e6d2-aa8b-4ee6-abaa-af8de2bb9d99 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff1ad378-b719-4704-b025-ffbee4e20edc) {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:41.097665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cb16e6d2-aa8b-4ee6-abaa-af8de2bb9d99 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65" status: 400 len: 269 microversion: 2.1 time: 246.702771 Sep 08 14:29:41.097665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:29:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (ip 10.222.0.25) !!! Sep 08 14:29:41.097665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:29:41 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (10.222.0.25) Sep 08 14:29:41.097665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-cb16e6d2-aa8b-4ee6-abaa-af8de2bb9d99 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Unhandled error: OSError: write error Sep 08 14:29:41.097665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:29:41.097665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:29:41.099148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1093/2472] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:25:34 2022] DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 => generated 0 bytes in 246704 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:29:41.099148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1cae1fd5-587e-442c-930d-e551ec9d16ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:41.099148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1cae1fd5-587e-442c-930d-e551ec9d16ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:41.100262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1cae1fd5-587e-442c-930d-e551ec9d16ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.101026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1cae1fd5-587e-442c-930d-e551ec9d16ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.101467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1cae1fd5-587e-442c-930d-e551ec9d16ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.102198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1cae1fd5-587e-442c-930d-e551ec9d16ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.102645 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1cae1fd5-587e-442c-930d-e551ec9d16ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.103063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1cae1fd5-587e-442c-930d-e551ec9d16ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.119701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-8ac31e35-822d-46c5-bb50-40eb05b76290 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0ae22db-017f-4a1e-bd57-524c4f24d42e) Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8ac31e35-822d-46c5-bb50-40eb05b76290 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Request method failure captured: Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 HTTP/1.1 Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfSsEI3OQCjaT8uW9n_C9Vd1UPOl7hGg4pp7-4nDKa37scg2VH8UGybcHZqyDn5ybTSjRS0vhV5EoGnyfNs_A0uLCWs9ZtiXuHPkrFMPvuwPxHSIVoo5s0Kpv1P1uWF7oFwsg1AH73mWHbrcQqseds7NX0AeiotGfcerp1SS6-JaE8A Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:29:41.124223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:29:41.126063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_179292c42f8e4041adc935c80dba0f70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:41.126063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:41.126063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:29:41.126063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:41.126063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:29:41.126063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:29:41.126063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:29:41.126063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 1f10a95e49744a6dbb73e54e00d81add Sep 08 14:29:41.126063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:29:41.126063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:29:41.126831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0ae22db-017f-4a1e-bd57-524c4f24d42e) Sep 08 14:29:41.126831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'server_id': 'd7ae43e4-11c8-417c-be73-4f9fb5c42f2c', 'id': '9a4342d7-3883-4385-bcd2-b4070743be65'} Sep 08 14:29:41.126831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:41.126831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.126831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:29:41.126831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:29:41.126831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:29:41.126831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:29:41.126831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:29:41.126831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:29:41.126831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:29:41.126831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:29:41.126831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:29:41.126831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:29:41.126831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:29:41.126831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:29:41.126831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0ae22db-017f-4a1e-bd57-524c4f24d42e) Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:29:41.128721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:29:41.130051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:29:41.130051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:29:41.130051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:29:41.130051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:29:41.130051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:29:41.130051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:29:41.130051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:29:41.130051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:29:41.130051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:29:41.130051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:29:41.130051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:29:41.130051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0ae22db-017f-4a1e-bd57-524c4f24d42e) Sep 08 14:29:41.130051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.130051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.130051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.130051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.130051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 08 14:29:41.133904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 08 14:29:41.133904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 14:29:41.133904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:29:41.133904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 08 14:29:41.133904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 08 14:29:41.133904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 08 14:29:41.133904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 08 14:29:41.133904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 08 14:29:41.133904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:29:41.133904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 08 14:29:41.133904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 08 14:29:41.133904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0ae22db-017f-4a1e-bd57-524c4f24d42e) Sep 08 14:29:41.133904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.133904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.133904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.133904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.133904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:41.133904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:41.137156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:41.137156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:41.137156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:41.137156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:41.137156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 08 14:29:41.137156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 08 14:29:41.137156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0ae22db-017f-4a1e-bd57-524c4f24d42e) Sep 08 14:29:41.137156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.137156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.137156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.137156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.137156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:41.137156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:41.137156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:41.137156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:41.137156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0ae22db-017f-4a1e-bd57-524c4f24d42e) Sep 08 14:29:41.137156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.140695 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1cae1fd5-587e-442c-930d-e551ec9d16ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:41.142525 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8ac31e35-822d-46c5-bb50-40eb05b76290 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a0ae22db-017f-4a1e-bd57-524c4f24d42e) {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:41.143403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8ac31e35-822d-46c5-bb50-40eb05b76290 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65" status: 400 len: 269 microversion: 2.1 time: 245.984605 Sep 08 14:29:41.143928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (ip 10.222.0.25) !!! Sep 08 14:29:41.143928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (10.222.0.25) Sep 08 14:29:41.144501 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-8ac31e35-822d-46c5-bb50-40eb05b76290 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Unhandled error: OSError: write error Sep 08 14:29:41.144501 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:29:41.144501 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:29:41.144781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1380/2473] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:25:35 2022] DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 => generated 0 bytes in 245986 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:29:41.217851 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8a42f1e6-a5b7-4b59-89ec-d024fff3a4fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:41.218386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8a42f1e6-a5b7-4b59-89ec-d024fff3a4fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:41.224370 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a42f1e6-a5b7-4b59-89ec-d024fff3a4fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.224870 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a42f1e6-a5b7-4b59-89ec-d024fff3a4fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.225301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a42f1e6-a5b7-4b59-89ec-d024fff3a4fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.225994 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a42f1e6-a5b7-4b59-89ec-d024fff3a4fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.226484 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a42f1e6-a5b7-4b59-89ec-d024fff3a4fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.226918 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8a42f1e6-a5b7-4b59-89ec-d024fff3a4fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.240869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-8a42f1e6-a5b7-4b59-89ec-d024fff3a4fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] HTTP exception thrown: Instance c14f6817-e3ab-4a36-ba23-590498bf1b25 could not be found. Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8a42f1e6-a5b7-4b59-89ec-d024fff3a4fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Request method failure captured: Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 HTTP/1.1 Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPpgseYqW3vfBhChnnflETRt0GDtjL3VmhITeNLuRSALZefRpZ4LlXgFeo5cygwUtHgGuuBrrkR39Uo-ZWGIlwSJVssX0SoJbjjdLhYxjZOutCiSryd_0UQo0F3jz05Y7DAMd5jNO7kUKOE8WKEghkWPfZegSJZuLFAO1D7uF_weRo Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Openstack-Nova-Api-Version: 2.20 Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:29:41.242471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:29:41.247033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:41.247033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:41.247033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:29:41.247033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:41.247033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:29:41.247033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:29:41.247033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:29:41.247033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:41.247033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:29:41.247033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:29:41.247033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance c14f6817-e3ab-4a36-ba23-590498bf1b25 could not be found. Sep 08 14:29:41.247033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'c14f6817-e3ab-4a36-ba23-590498bf1b25'} Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c14f6817-e3ab-4a36-ba23-590498bf1b25 could not be found. Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:41.247953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c14f6817-e3ab-4a36-ba23-590498bf1b25 could not be found. Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c14f6817-e3ab-4a36-ba23-590498bf1b25 could not be found. Sep 08 14:29:41.250321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.252279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8a42f1e6-a5b7-4b59-89ec-d024fff3a4fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Returning 404 to user: Instance c14f6817-e3ab-4a36-ba23-590498bf1b25 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:41.252279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8a42f1e6-a5b7-4b59-89ec-d024fff3a4fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 404 len: 111 microversion: 2.20 time: 0.098366 Sep 08 14:29:41.252279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 (ip 10.222.0.25) !!! Sep 08 14:29:41.252279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 (10.222.0.25) Sep 08 14:29:41.252279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-8a42f1e6-a5b7-4b59-89ec-d024fff3a4fb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Unhandled error: OSError: write error Sep 08 14:29:41.252279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:29:41.252279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:29:41.252279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1381/2474] 10.222.0.25 () {60 vars in 1280 bytes} [Thu Sep 8 14:29:41 2022] DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 0 bytes in 100 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:29:41.253101 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.compute.api [None req-1cae1fd5-587e-442c-930d-e551ec9d16ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Instance is already in deleting state, ignoring this request Sep 08 14:29:41.253267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5bc09395-c18f-47a1-b717-2ed8e4761cc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:41.253580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1cae1fd5-587e-442c-930d-e551ec9d16ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 204 len: 0 microversion: 2.1 time: 0.163626 Sep 08 14:29:41.253887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5bc09395-c18f-47a1-b717-2ed8e4761cc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:41.254004 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:29:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (ip 10.222.0.25) !!! Sep 08 14:29:41.254004 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:29:41 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (10.222.0.25) Sep 08 14:29:41.254004 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-1cae1fd5-587e-442c-930d-e551ec9d16ed tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Unhandled error: OSError: write error Sep 08 14:29:41.254004 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:29:41.254004 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:29:41.254004 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1094/2475] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:41 2022] DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:29:41.256850 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5bc09395-c18f-47a1-b717-2ed8e4761cc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.257353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5bc09395-c18f-47a1-b717-2ed8e4761cc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.257806 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5bc09395-c18f-47a1-b717-2ed8e4761cc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.258543 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5bc09395-c18f-47a1-b717-2ed8e4761cc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.258985 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5bc09395-c18f-47a1-b717-2ed8e4761cc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.259421 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5bc09395-c18f-47a1-b717-2ed8e4761cc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.285279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5bc09395-c18f-47a1-b717-2ed8e4761cc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:41.353595 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-43698367-b9c6-4cf4-aa91-c79568b38125 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '.version_select at 0x7fce059a5f70>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:41.376706 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-43698367-b9c6-4cf4-aa91-c79568b38125 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-43698367-b9c6-4cf4-aa91-c79568b38125 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 HTTP/1.1 Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:29:41.378502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:29:41.380273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:41.380273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:41.380273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:29:41.380273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:41.380273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:29:41.380273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:29:41.380273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:29:41.380273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:29:41.380273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:29:41.380273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 14:29:41.380273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:41.380273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'tempest-keypair-997401276'} Sep 08 14:29:41.380273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:29:41.381359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:41.383014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:41.384636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:41.384636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:41.384636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 08 14:29:41.384636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 08 14:29:41.384636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:29:41.384636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:29:41.384636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:41.384636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.384636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.384636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.384636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.384636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:41.384636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:41.384636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:41.384636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:41.384636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:41.384636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.386094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-43698367-b9c6-4cf4-aa91-c79568b38125 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:41.386094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-43698367-b9c6-4cf4-aa91-c79568b38125 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276" status: 404 len: 131 microversion: 2.1 time: 0.127484 Sep 08 14:29:41.386094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:29:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-keypair-997401276 (ip 10.222.0.25) !!! Sep 08 14:29:41.386094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:29:41 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 (10.222.0.25) Sep 08 14:29:41.386094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-43698367-b9c6-4cf4-aa91-c79568b38125 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Unhandled error: OSError: write error Sep 08 14:29:41.386094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:29:41.386094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:29:41.386094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1095/2476] 10.222.0.25 () {58 vars in 1218 bytes} [Thu Sep 8 14:29:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 => generated 0 bytes in 129 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:29:41.387093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8fae4908-ce7e-4791-b92d-e7928b442881 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:41.387093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8fae4908-ce7e-4791-b92d-e7928b442881 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:41.397404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.compute.api [None req-5bc09395-c18f-47a1-b717-2ed8e4761cc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Instance is already in deleting state, ignoring this request Sep 08 14:29:41.398303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5bc09395-c18f-47a1-b717-2ed8e4761cc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 204 len: 0 microversion: 2.1 time: 0.152377 Sep 08 14:29:41.399430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 (ip 10.222.0.25) !!! Sep 08 14:29:41.399430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 (10.222.0.25) Sep 08 14:29:41.400020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-5bc09395-c18f-47a1-b717-2ed8e4761cc3 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Unhandled error: OSError: write error Sep 08 14:29:41.400020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:29:41.400020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:29:41.400485 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1382/2477] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:41 2022] DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:29:41.401693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fae4908-ce7e-4791-b92d-e7928b442881 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.402659 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fae4908-ce7e-4791-b92d-e7928b442881 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.403703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fae4908-ce7e-4791-b92d-e7928b442881 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.404739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f39ed5f5-d537-4c94-8afe-348330645d58 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:41.405234 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f39ed5f5-d537-4c94-8afe-348330645d58 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:41.405462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fae4908-ce7e-4791-b92d-e7928b442881 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.406649 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fae4908-ce7e-4791-b92d-e7928b442881 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.407494 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8fae4908-ce7e-4791-b92d-e7928b442881 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.414425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f39ed5f5-d537-4c94-8afe-348330645d58 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.414425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f39ed5f5-d537-4c94-8afe-348330645d58 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.414425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f39ed5f5-d537-4c94-8afe-348330645d58 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.414425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f39ed5f5-d537-4c94-8afe-348330645d58 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.414829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f39ed5f5-d537-4c94-8afe-348330645d58 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.414829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f39ed5f5-d537-4c94-8afe-348330645d58 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.443998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f39ed5f5-d537-4c94-8afe-348330645d58 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:41.488971 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.compute.api [None req-f39ed5f5-d537-4c94-8afe-348330645d58 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Instance is already in deleting state, ignoring this request Sep 08 14:29:41.489793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f39ed5f5-d537-4c94-8afe-348330645d58 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 204 len: 0 microversion: 2.1 time: 0.088775 Sep 08 14:29:41.492311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (ip 10.222.0.25) !!! Sep 08 14:29:41.492311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (10.222.0.25) Sep 08 14:29:41.492311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-f39ed5f5-d537-4c94-8afe-348330645d58 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Unhandled error: OSError: write error Sep 08 14:29:41.492311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:29:41.492311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:29:41.492311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1383/2478] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:41 2022] DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 0 bytes in 90 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:29:41.495526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9d26f9aa-0c06-4be5-ae24-96b3f1dc07c7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '.version_select at 0x7fce05946f70>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:41.500130 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-8fae4908-ce7e-4791-b92d-e7928b442881 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lazy-loading 'services' on Instance uuid d7ae43e4-11c8-417c-be73-4f9fb5c42f2c {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:29:41.551448 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-8fae4908-ce7e-4791-b92d-e7928b442881 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7d477567-4daf-49e6-8426-5fba9c0a91d3) Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8fae4908-ce7e-4791-b92d-e7928b442881 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Request method failure captured: Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 HTTP/1.1 Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfSsEI3OQCjaT8uW9n_C9Vd1UPOl7hGg4pp7-4nDKa37scg2VH8UGybcHZqyDn5ybTSjRS0vhV5EoGnyfNs_A0uLCWs9ZtiXuHPkrFMPvuwPxHSIVoo5s0Kpv1P1uWF7oFwsg1AH73mWHbrcQqseds7NX0AeiotGfcerp1SS6-JaE8A Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:29:41.553607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:29:41.555642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_179292c42f8e4041adc935c80dba0f70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:41.555642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:41.555642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:29:41.555642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:41.555642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:29:41.555642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:29:41.555642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:29:41.555642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 1f10a95e49744a6dbb73e54e00d81add Sep 08 14:29:41.555642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:29:41.555642 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:29:41.556375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7d477567-4daf-49e6-8426-5fba9c0a91d3) Sep 08 14:29:41.556375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'server_id': 'd7ae43e4-11c8-417c-be73-4f9fb5c42f2c', 'id': '9a4342d7-3883-4385-bcd2-b4070743be65'} Sep 08 14:29:41.556375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:41.556375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.556375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:29:41.556375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:29:41.556375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:29:41.556375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:29:41.556375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:29:41.556375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:29:41.556375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:29:41.556375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:29:41.556375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:29:41.556375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:29:41.556375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:29:41.556375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:29:41.556375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7d477567-4daf-49e6-8426-5fba9c0a91d3) Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:29:41.558192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 08 14:29:41.560478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 08 14:29:41.560478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 08 14:29:41.560478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 08 14:29:41.560478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 08 14:29:41.560478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 08 14:29:41.560478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 08 14:29:41.560478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 08 14:29:41.560478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:29:41.560478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:29:41.560478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:29:41.560478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:29:41.560478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7d477567-4daf-49e6-8426-5fba9c0a91d3) Sep 08 14:29:41.560478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.560478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.560478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.560478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.560478 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 08 14:29:41.562096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 08 14:29:41.562096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 08 14:29:41.562096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:29:41.562096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 08 14:29:41.562096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 08 14:29:41.562096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 08 14:29:41.562096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 08 14:29:41.562096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 08 14:29:41.562096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 08 14:29:41.562096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 08 14:29:41.562096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 08 14:29:41.562096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7d477567-4daf-49e6-8426-5fba9c0a91d3) Sep 08 14:29:41.562096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.562096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.562096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.562096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.562096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:41.562096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:41.564671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:41.564671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:41.564671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:41.564671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:41.564671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 08 14:29:41.564671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 08 14:29:41.564671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7d477567-4daf-49e6-8426-5fba9c0a91d3) Sep 08 14:29:41.564671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.564671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.564671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.564671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.564671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:41.564671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:41.564671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:41.564671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:41.564671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7d477567-4daf-49e6-8426-5fba9c0a91d3) Sep 08 14:29:41.564671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.566302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8fae4908-ce7e-4791-b92d-e7928b442881 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7d477567-4daf-49e6-8426-5fba9c0a91d3) {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:41.566302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8fae4908-ce7e-4791-b92d-e7928b442881 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65" status: 400 len: 269 microversion: 2.1 time: 0.173114 Sep 08 14:29:41.566302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:29:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (ip 10.222.0.25) !!! Sep 08 14:29:41.566302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:29:41 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (10.222.0.25) Sep 08 14:29:41.566302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-8fae4908-ce7e-4791-b92d-e7928b442881 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Unhandled error: OSError: write error Sep 08 14:29:41.566302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:29:41.566302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:29:41.566921 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1096/2479] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:29:41 2022] DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 => generated 0 bytes in 175 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:29:41.566921 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4cbd9036-57ce-4dcd-a551-56e39b1a3259 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:41.566921 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4cbd9036-57ce-4dcd-a551-56e39b1a3259 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:41.568086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4cbd9036-57ce-4dcd-a551-56e39b1a3259 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.568570 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4cbd9036-57ce-4dcd-a551-56e39b1a3259 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.568996 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4cbd9036-57ce-4dcd-a551-56e39b1a3259 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.569730 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4cbd9036-57ce-4dcd-a551-56e39b1a3259 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.572957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4cbd9036-57ce-4dcd-a551-56e39b1a3259 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.572957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4cbd9036-57ce-4dcd-a551-56e39b1a3259 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.574344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9d26f9aa-0c06-4be5-ae24-96b3f1dc07c7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013" status: 204 len: 0 microversion: 2.20 time: 0.081152 Sep 08 14:29:41.574344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-keypair-920229013 (ip 10.222.0.25) !!! Sep 08 14:29:41.574344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013 (10.222.0.25) Sep 08 14:29:41.577826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-9d26f9aa-0c06-4be5-ae24-96b3f1dc07c7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Unhandled error: OSError: write error Sep 08 14:29:41.577826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:29:41.577826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:29:41.577826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1384/2480] 10.222.0.25 () {60 vars in 1259 bytes} [Thu Sep 8 14:29:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:29:41.599654 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4cbd9036-57ce-4dcd-a551-56e39b1a3259 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:41.642602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-5b3ded9c-679a-4cf1-8e66-9f4f196c0d97 req-09a48a23-cec6-41b2-942a-531d5a35823e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d1062abf-7525-48c5-a9cb-91da3855fd1b", "name": "network-vif-deleted", "tag": "2d04f3d2-99c9-4712-bc13-6ebdffcb949f"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:29:41.656966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-5b3ded9c-679a-4cf1-8e66-9f4f196c0d97 req-09a48a23-cec6-41b2-942a-531d5a35823e service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.657458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-5b3ded9c-679a-4cf1-8e66-9f4f196c0d97 req-09a48a23-cec6-41b2-942a-531d5a35823e service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.657892 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-5b3ded9c-679a-4cf1-8e66-9f4f196c0d97 req-09a48a23-cec6-41b2-942a-531d5a35823e service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.671567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-5b3ded9c-679a-4cf1-8e66-9f4f196c0d97 req-09a48a23-cec6-41b2-942a-531d5a35823e service nova] Creating event network-vif-deleted:2d04f3d2-99c9-4712-bc13-6ebdffcb949f for instance d1062abf-7525-48c5-a9cb-91da3855fd1b on hv-cinder-85646 Sep 08 14:29:41.676726 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-5b3ded9c-679a-4cf1-8e66-9f4f196c0d97 req-09a48a23-cec6-41b2-942a-531d5a35823e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.100804 Sep 08 14:29:41.677306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.25) !!! Sep 08 14:29:41.677306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during POST /compute/v2.1/os-server-external-events (10.222.0.25) Sep 08 14:29:41.678002 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [req-5b3ded9c-679a-4cf1-8e66-9f4f196c0d97 req-09a48a23-cec6-41b2-942a-531d5a35823e service nova] Unhandled error: OSError: write error Sep 08 14:29:41.678002 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:29:41.678002 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:29:41.678355 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1385/2481] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:29:41 2022] POST /compute/v2.1/os-server-external-events => generated 0 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:29:41.683275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f8fd240e-6f93-4729-bd5c-5ab5694a7dff tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '.version_select at 0x7fce021f2670>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:41.696966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-f8fd240e-6f93-4729-bd5c-5ab5694a7dff tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f8fd240e-6f93-4729-bd5c-5ab5694a7dff tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 HTTP/1.1 Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:29:41.698884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:29:41.706156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:41.706156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:41.706156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:29:41.706156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:41.706156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:29:41.706156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:29:41.706156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:29:41.706156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:29:41.706156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:29:41.706156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: Sep 08 14:29:41.706156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:41.706156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'tempest-keypair-997401276'} Sep 08 14:29:41.706156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:29:41.707117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:41.708426 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:41.709741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:41.709741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:41.709741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 08 14:29:41.709741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 08 14:29:41.709741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:29:41.709741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:29:41.709741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:41.709741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.709741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.709741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.709741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.709741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:41.709741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:41.709741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:41.709741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:41.709741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:41.709741 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.711008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f8fd240e-6f93-4729-bd5c-5ab5694a7dff tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:41.711008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f8fd240e-6f93-4729-bd5c-5ab5694a7dff tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276" status: 404 len: 131 microversion: 2.1 time: 0.021090 Sep 08 14:29:41.711008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-keypair-997401276 (ip 10.222.0.25) !!! Sep 08 14:29:41.711008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 (10.222.0.25) Sep 08 14:29:41.711008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-f8fd240e-6f93-4729-bd5c-5ab5694a7dff tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Unhandled error: OSError: write error Sep 08 14:29:41.711008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:29:41.711008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:29:41.711008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1386/2482] 10.222.0.25 () {58 vars in 1218 bytes} [Thu Sep 8 14:29:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 => generated 0 bytes in 23 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:29:41.712822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2ca6792e-866f-44d1-b875-2a83a5c1026e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:41.712822 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2ca6792e-866f-44d1-b875-2a83a5c1026e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:41.717111 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ca6792e-866f-44d1-b875-2a83a5c1026e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.717609 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ca6792e-866f-44d1-b875-2a83a5c1026e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.718054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ca6792e-866f-44d1-b875-2a83a5c1026e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.719046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ca6792e-866f-44d1-b875-2a83a5c1026e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.719266 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ca6792e-866f-44d1-b875-2a83a5c1026e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.719705 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2ca6792e-866f-44d1-b875-2a83a5c1026e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.764109 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.compute.api [None req-4cbd9036-57ce-4dcd-a551-56e39b1a3259 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Instance is already in deleting state, ignoring this request Sep 08 14:29:41.765014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4cbd9036-57ce-4dcd-a551-56e39b1a3259 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 204 len: 0 microversion: 2.1 time: 0.207863 Sep 08 14:29:41.765555 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:29:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 (ip 10.222.0.25) !!! Sep 08 14:29:41.765555 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:29:41 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 (10.222.0.25) Sep 08 14:29:41.766112 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-4cbd9036-57ce-4dcd-a551-56e39b1a3259 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Unhandled error: OSError: write error Sep 08 14:29:41.766112 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:29:41.766112 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:29:41.766406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1097/2483] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:41 2022] DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:29:41.770637 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ca0fc3b6-d084-4e84-b93a-05eaa16d0d83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:41.771121 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ca0fc3b6-d084-4e84-b93a-05eaa16d0d83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:41.776691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca0fc3b6-d084-4e84-b93a-05eaa16d0d83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.777167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca0fc3b6-d084-4e84-b93a-05eaa16d0d83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.777629 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca0fc3b6-d084-4e84-b93a-05eaa16d0d83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.778859 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca0fc3b6-d084-4e84-b93a-05eaa16d0d83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.779300 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca0fc3b6-d084-4e84-b93a-05eaa16d0d83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.779721 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ca0fc3b6-d084-4e84-b93a-05eaa16d0d83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.805585 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ca0fc3b6-d084-4e84-b93a-05eaa16d0d83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:41.813934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-2ca6792e-866f-44d1-b875-2a83a5c1026e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] HTTP exception thrown: Instance d7ae43e4-11c8-417c-be73-4f9fb5c42f2c is not attached to volume 9a4342d7-3883-4385-bcd2-b4070743be65 Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2ca6792e-866f-44d1-b875-2a83a5c1026e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Request method failure captured: Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 HTTP/1.1 Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfSsEI3OQCjaT8uW9n_C9Vd1UPOl7hGg4pp7-4nDKa37scg2VH8UGybcHZqyDn5ybTSjRS0vhV5EoGnyfNs_A0uLCWs9ZtiXuHPkrFMPvuwPxHSIVoo5s0Kpv1P1uWF7oFwsg1AH73mWHbrcQqseds7NX0AeiotGfcerp1SS6-JaE8A Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:29:41.815812 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:29:41.817347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_179292c42f8e4041adc935c80dba0f70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:41.817347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:41.817347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:29:41.817347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:41.817347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:29:41.817347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:29:41.817347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:29:41.817347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 1f10a95e49744a6dbb73e54e00d81add Sep 08 14:29:41.817347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:29:41.817347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:29:41.817347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance d7ae43e4-11c8-417c-be73-4f9fb5c42f2c is not attached to volume 9a4342d7-3883-4385-bcd2-b4070743be65 Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'server_id': 'd7ae43e4-11c8-417c-be73-4f9fb5c42f2c', 'id': '9a4342d7-3883-4385-bcd2-b4070743be65'} Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id 9a4342d7-3883-4385-bcd2-b4070743be65. Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:41.818181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:41.819843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:41.819843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Sep 08 14:29:41.819843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 08 14:29:41.819843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d7ae43e4-11c8-417c-be73-4f9fb5c42f2c is not attached to volume 9a4342d7-3883-4385-bcd2-b4070743be65 Sep 08 14:29:41.819843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.819843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.819843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.819843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.819843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:41.819843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:41.819843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:41.819843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:41.819843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d7ae43e4-11c8-417c-be73-4f9fb5c42f2c is not attached to volume 9a4342d7-3883-4385-bcd2-b4070743be65 Sep 08 14:29:41.819843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.819843 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2ca6792e-866f-44d1-b875-2a83a5c1026e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Returning 404 to user: Instance d7ae43e4-11c8-417c-be73-4f9fb5c42f2c is not attached to volume 9a4342d7-3883-4385-bcd2-b4070743be65 {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:41.820954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2ca6792e-866f-44d1-b875-2a83a5c1026e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65" status: 404 len: 154 microversion: 2.1 time: 0.114799 Sep 08 14:29:41.820954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (ip 10.222.0.25) !!! Sep 08 14:29:41.820954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 (10.222.0.25) Sep 08 14:29:41.820954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-2ca6792e-866f-44d1-b875-2a83a5c1026e tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Unhandled error: OSError: write error Sep 08 14:29:41.820954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:29:41.820954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:29:41.820954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1387/2484] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:29:41 2022] DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 => generated 0 bytes in 116 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:29:41.822949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-91eebd34-73b4-4607-a16f-070e44af1483 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '.version_select at 0x7fce01c995e0>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:41.837199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-91eebd34-73b4-4607-a16f-070e44af1483 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] HTTP exception thrown: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-91eebd34-73b4-4607-a16f-070e44af1483 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Request method failure captured: Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013 HTTP/1.1 Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPpgseYqW3vfBhChnnflETRt0GDtjL3VmhITeNLuRSALZefRpZ4LlXgFeo5cygwUtHgGuuBrrkR39Uo-ZWGIlwSJVssX0SoJbjjdLhYxjZOutCiSryd_0UQo0F3jz05Y7DAMd5jNO7kUKOE8WKEghkWPfZegSJZuLFAO1D7uF_weRo Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Openstack-Nova-Api-Version: 2.20 Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:29:41.839124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:29:41.841503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:41.841503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:41.841503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:29:41.841503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:41.841503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:29:41.841503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:29:41.841503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:29:41.841503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:41.841503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:29:41.841503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: Sep 08 14:29:41.841503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:41.841503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'tempest-keypair-920229013'} Sep 08 14:29:41.841503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:29:41.842757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:41.844260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 160, in delete Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._delete(req, id, user_id=user_id) Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:41.845603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.847377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-91eebd34-73b4-4607-a16f-070e44af1483 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Returning 404 to user: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:41.847377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-91eebd34-73b4-4607-a16f-070e44af1483 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013" status: 404 len: 131 microversion: 2.20 time: 0.021229 Sep 08 14:29:41.847377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-keypair-920229013 (ip 10.222.0.25) !!! Sep 08 14:29:41.847377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013 (10.222.0.25) Sep 08 14:29:41.847377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-91eebd34-73b4-4607-a16f-070e44af1483 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Unhandled error: OSError: write error Sep 08 14:29:41.847377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:29:41.847377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:29:41.847377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1388/2485] 10.222.0.25 () {60 vars in 1259 bytes} [Thu Sep 8 14:29:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013 => generated 0 bytes in 23 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:29:41.848110 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4474bf70-0188-4ba4-b650-69b2a74cfdce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:41.848376 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4474bf70-0188-4ba4-b650-69b2a74cfdce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:41.854590 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4474bf70-0188-4ba4-b650-69b2a74cfdce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.855087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4474bf70-0188-4ba4-b650-69b2a74cfdce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.855531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4474bf70-0188-4ba4-b650-69b2a74cfdce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.856907 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4474bf70-0188-4ba4-b650-69b2a74cfdce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.857354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4474bf70-0188-4ba4-b650-69b2a74cfdce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.857784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4474bf70-0188-4ba4-b650-69b2a74cfdce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.883449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4474bf70-0188-4ba4-b650-69b2a74cfdce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:41.936260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.compute.api [None req-4474bf70-0188-4ba4-b650-69b2a74cfdce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Instance is already in deleting state, ignoring this request Sep 08 14:29:41.936260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4474bf70-0188-4ba4-b650-69b2a74cfdce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 204 len: 0 microversion: 2.1 time: 0.092139 Sep 08 14:29:41.937568 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 (ip 10.222.0.25) !!! Sep 08 14:29:41.937568 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:29:41 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 (10.222.0.25) Sep 08 14:29:41.938374 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: CRITICAL nova [None req-4474bf70-0188-4ba4-b650-69b2a74cfdce tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Unhandled error: OSError: write error Sep 08 14:29:41.938374 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova OSError: write error Sep 08 14:29:41.938374 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova Sep 08 14:29:41.938374 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1389/2486] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:41 2022] DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:29:41.944896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-900ea95c-82ba-47c1-b501-717b52592590 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '.version_select at 0x7fce058cfdc0>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:41.958130 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-900ea95c-82ba-47c1-b501-717b52592590 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-900ea95c-82ba-47c1-b501-717b52592590 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 HTTP/1.1 Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:29:41.959949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:29:41.961509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:41.961509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:41.961509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:29:41.961509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:41.961509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:29:41.961509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:29:41.961509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:29:41.961509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:29:41.961509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:29:41.961509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: Sep 08 14:29:41.961509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:41.961509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'tempest-keypair-997401276'} Sep 08 14:29:41.961509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:29:41.976009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:41.977760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:41.979920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:41.979920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:41.979920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 08 14:29:41.979920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 08 14:29:41.979920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:29:41.979920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:29:41.979920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:41.979920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.979920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:41.979920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.979920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:41.979920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:41.979920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:41.979920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:41.979920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:41.979920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:41.979920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:41.981362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-900ea95c-82ba-47c1-b501-717b52592590 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:41.981362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-900ea95c-82ba-47c1-b501-717b52592590 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276" status: 404 len: 131 microversion: 2.1 time: 0.021714 Sep 08 14:29:41.981362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1390/2487] 10.222.0.25 () {58 vars in 1218 bytes} [Thu Sep 8 14:29:41 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 => generated 131 bytes in 23 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:29:41.981362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-eb7de989-b42c-4ffd-8cff-c8b835387faf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:41.981362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-eb7de989-b42c-4ffd-8cff-c8b835387faf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:41.981834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb7de989-b42c-4ffd-8cff-c8b835387faf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.981834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb7de989-b42c-4ffd-8cff-c8b835387faf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.981834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb7de989-b42c-4ffd-8cff-c8b835387faf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:41.981834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb7de989-b42c-4ffd-8cff-c8b835387faf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:41.982219 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb7de989-b42c-4ffd-8cff-c8b835387faf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:41.982219 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eb7de989-b42c-4ffd-8cff-c8b835387faf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.028236 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-eb7de989-b42c-4ffd-8cff-c8b835387faf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] HTTP exception thrown: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-eb7de989-b42c-4ffd-8cff-c8b835387faf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Request method failure captured: Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 HTTP/1.1 Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfSsEI3OQCjaT8uW9n_C9Vd1UPOl7hGg4pp7-4nDKa37scg2VH8UGybcHZqyDn5ybTSjRS0vhV5EoGnyfNs_A0uLCWs9ZtiXuHPkrFMPvuwPxHSIVoo5s0Kpv1P1uWF7oFwsg1AH73mWHbrcQqseds7NX0AeiotGfcerp1SS6-JaE8A Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:29:42.031615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:29:42.035395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_179292c42f8e4041adc935c80dba0f70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:42.035395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:42.035395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:29:42.035395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:42.035395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:29:42.035395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:29:42.035395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:29:42.035395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 1f10a95e49744a6dbb73e54e00d81add Sep 08 14:29:42.035395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:29:42.035395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:29:42.035395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:42.036218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'server_id': 'd7ae43e4-11c8-417c-be73-4f9fb5c42f2c', 'id': '9a4342d7-3883-4385-bcd2-b4070743be65'} Sep 08 14:29:42.036218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:42.036218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.036218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:29:42.036218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:29:42.036218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 08 14:29:42.036218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 08 14:29:42.036218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 08 14:29:42.036218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 08 14:29:42.036218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 08 14:29:42.036218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 08 14:29:42.036218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 08 14:29:42.036218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 08 14:29:42.036218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:29:42.036218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:29:42.036218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:29:42.036218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:29:42.036218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. (HTTP 404) (Request-ID: req-be4a7f27-3d28-49ee-9cb2-b4cd2a7a6877) Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 547, in delete Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi volume = self.volume_api.get(context, volume_id) Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 08 14:29:42.037580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 549, in delete Sep 08 14:29:42.039109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:29:42.041069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:42.041069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.041069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.041069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.041069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.041069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.041069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.041069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:42.041069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:42.041069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:42.041069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.041069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-eb7de989-b42c-4ffd-8cff-c8b835387faf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Returning 404 to user: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:42.041069 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-eb7de989-b42c-4ffd-8cff-c8b835387faf tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65" status: 404 len: 109 microversion: 2.1 time: 0.070209 Sep 08 14:29:42.042013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1391/2488] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:29:41 2022] DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 => generated 109 bytes in 71 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:29:42.042013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-84fc9b60-e715-412e-b871-0bfd0333cd18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.043059 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-84fc9b60-e715-412e-b871-0bfd0333cd18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:42.051853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84fc9b60-e715-412e-b871-0bfd0333cd18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.052381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84fc9b60-e715-412e-b871-0bfd0333cd18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.052830 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84fc9b60-e715-412e-b871-0bfd0333cd18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.053746 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84fc9b60-e715-412e-b871-0bfd0333cd18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.054019 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84fc9b60-e715-412e-b871-0bfd0333cd18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.054535 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84fc9b60-e715-412e-b871-0bfd0333cd18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.060131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ca0fc3b6-d084-4e84-b93a-05eaa16d0d83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 204 len: 0 microversion: 2.1 time: 0.292614 Sep 08 14:29:42.060964 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:29:42 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (ip 10.222.0.25) !!! Sep 08 14:29:42.060964 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:29:42 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b (10.222.0.25) Sep 08 14:29:42.061323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: CRITICAL nova [None req-ca0fc3b6-d084-4e84-b93a-05eaa16d0d83 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Unhandled error: OSError: write error Sep 08 14:29:42.061323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova OSError: write error Sep 08 14:29:42.061323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova Sep 08 14:29:42.061826 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1098/2489] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:41 2022] DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 0 bytes in 295 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 08 14:29:42.067371 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-406aaeda-da73-47f6-a7d3-5d245007a268 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '.version_select at 0x7fce0589e8b0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.081699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-406aaeda-da73-47f6-a7d3-5d245007a268 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] HTTP exception thrown: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-406aaeda-da73-47f6-a7d3-5d245007a268 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Request method failure captured: Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013 HTTP/1.1 Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPpgseYqW3vfBhChnnflETRt0GDtjL3VmhITeNLuRSALZefRpZ4LlXgFeo5cygwUtHgGuuBrrkR39Uo-ZWGIlwSJVssX0SoJbjjdLhYxjZOutCiSryd_0UQo0F3jz05Y7DAMd5jNO7kUKOE8WKEghkWPfZegSJZuLFAO1D7uF_weRo Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Openstack-Nova-Api-Version: 2.20 Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:29:42.083518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:29:42.085142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:42.085142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:42.085142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:29:42.085142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:42.085142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:29:42.085142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:29:42.085142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:29:42.085142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:42.085142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:29:42.085142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 14:29:42.085142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:42.085142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'tempest-keypair-920229013'} Sep 08 14:29:42.085142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:29:42.086439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:42.088616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 160, in delete Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req, id, user_id=user_id) Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:42.091601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.094531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-406aaeda-da73-47f6-a7d3-5d245007a268 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Returning 404 to user: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:42.094531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-406aaeda-da73-47f6-a7d3-5d245007a268 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013" status: 404 len: 131 microversion: 2.20 time: 0.021775 Sep 08 14:29:42.094531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1099/2490] 10.222.0.25 () {60 vars in 1259 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013 => generated 131 bytes in 23 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 08 14:29:42.094531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-48e89de4-bf2f-4d5e-b103-ed6cf8b1c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.094531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-48e89de4-bf2f-4d5e-b103-ed6cf8b1c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:42.095297 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-84fc9b60-e715-412e-b871-0bfd0333cd18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:42.099299 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-48e89de4-bf2f-4d5e-b103-ed6cf8b1c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.108789 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-48e89de4-bf2f-4d5e-b103-ed6cf8b1c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.109312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-48e89de4-bf2f-4d5e-b103-ed6cf8b1c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.114419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-48e89de4-bf2f-4d5e-b103-ed6cf8b1c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.115155 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-48e89de4-bf2f-4d5e-b103-ed6cf8b1c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.115606 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-48e89de4-bf2f-4d5e-b103-ed6cf8b1c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.145680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.compute.api [None req-84fc9b60-e715-412e-b871-0bfd0333cd18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Instance is already in deleting state, ignoring this request Sep 08 14:29:42.146664 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-84fc9b60-e715-412e-b871-0bfd0333cd18 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 204 len: 0 microversion: 2.1 time: 0.110662 Sep 08 14:29:42.148341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1392/2491] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:29:42.150468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-48e89de4-bf2f-4d5e-b103-ed6cf8b1c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:42.152658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8ff99d39-e1b7-431f-8a20-e38dc3f08d66 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '.version_select at 0x7fce05946430>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.167742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-8ff99d39-e1b7-431f-8a20-e38dc3f08d66 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8ff99d39-e1b7-431f-8a20-e38dc3f08d66 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 HTTP/1.1 Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:29:42.169952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:29:42.171702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:42.171702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:42.171702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:29:42.171702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:42.171702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:29:42.171702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:29:42.171702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:29:42.171702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:29:42.171702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:29:42.171702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: Sep 08 14:29:42.171702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:42.171702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'tempest-keypair-997401276'} Sep 08 14:29:42.171702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:29:42.172948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:42.174943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:42.177811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:42.177811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:42.177811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 08 14:29:42.177811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 08 14:29:42.177811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:29:42.177811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:29:42.177811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:42.177811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.177811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.177811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.177811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.177811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.177811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.177811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:42.177811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:42.177811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:42.177811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.184012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8ff99d39-e1b7-431f-8a20-e38dc3f08d66 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:42.184012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8ff99d39-e1b7-431f-8a20-e38dc3f08d66 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276" status: 404 len: 131 microversion: 2.1 time: 0.022563 Sep 08 14:29:42.184012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1393/2492] 10.222.0.25 () {58 vars in 1218 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 => generated 131 bytes in 24 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:29:42.184012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c99624c7-a5ce-4940-ae6a-d4c43c353efd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.184012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c99624c7-a5ce-4940-ae6a-d4c43c353efd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:42.185644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c99624c7-a5ce-4940-ae6a-d4c43c353efd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.186879 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c99624c7-a5ce-4940-ae6a-d4c43c353efd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.187375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c99624c7-a5ce-4940-ae6a-d4c43c353efd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.188080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c99624c7-a5ce-4940-ae6a-d4c43c353efd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.188536 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c99624c7-a5ce-4940-ae6a-d4c43c353efd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.189065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c99624c7-a5ce-4940-ae6a-d4c43c353efd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.203977 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.compute.api [None req-48e89de4-bf2f-4d5e-b103-ed6cf8b1c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Instance is already in deleting state, ignoring this request Sep 08 14:29:42.204912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-48e89de4-bf2f-4d5e-b103-ed6cf8b1c770 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 204 len: 0 microversion: 2.1 time: 0.118006 Sep 08 14:29:42.205577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1100/2493] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:29:42.210444 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0bb41242-5ffe-4258-9c92-25254becfc25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.210935 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0bb41242-5ffe-4258-9c92-25254becfc25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:42.217564 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0bb41242-5ffe-4258-9c92-25254becfc25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.218046 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0bb41242-5ffe-4258-9c92-25254becfc25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.218512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0bb41242-5ffe-4258-9c92-25254becfc25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.219289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0bb41242-5ffe-4258-9c92-25254becfc25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.219737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0bb41242-5ffe-4258-9c92-25254becfc25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.220236 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0bb41242-5ffe-4258-9c92-25254becfc25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.245388 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0bb41242-5ffe-4258-9c92-25254becfc25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:42.245553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-c99624c7-a5ce-4940-ae6a-d4c43c353efd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] HTTP exception thrown: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c99624c7-a5ce-4940-ae6a-d4c43c353efd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Request method failure captured: Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 HTTP/1.1 Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfSsEI3OQCjaT8uW9n_C9Vd1UPOl7hGg4pp7-4nDKa37scg2VH8UGybcHZqyDn5ybTSjRS0vhV5EoGnyfNs_A0uLCWs9ZtiXuHPkrFMPvuwPxHSIVoo5s0Kpv1P1uWF7oFwsg1AH73mWHbrcQqseds7NX0AeiotGfcerp1SS6-JaE8A Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:29:42.247503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:29:42.249696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_179292c42f8e4041adc935c80dba0f70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:42.249696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:42.249696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:29:42.249696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:42.249696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:29:42.249696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:29:42.249696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:29:42.249696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 1f10a95e49744a6dbb73e54e00d81add Sep 08 14:29:42.249696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:29:42.249696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:29:42.249696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:42.250602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'server_id': 'd7ae43e4-11c8-417c-be73-4f9fb5c42f2c', 'id': '9a4342d7-3883-4385-bcd2-b4070743be65'} Sep 08 14:29:42.250602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:42.250602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.250602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:29:42.250602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:29:42.250602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 08 14:29:42.250602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 08 14:29:42.250602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 08 14:29:42.250602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 08 14:29:42.250602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 08 14:29:42.250602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 08 14:29:42.250602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 08 14:29:42.250602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 08 14:29:42.250602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:29:42.250602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:29:42.250602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:29:42.250602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:29:42.250602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. (HTTP 404) (Request-ID: req-b7c6afbe-e9de-4808-825e-1958f2c4f7fb) Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 547, in delete Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi volume = self.volume_api.get(context, volume_id) Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 08 14:29:42.251861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 549, in delete Sep 08 14:29:42.253322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:29:42.254665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:42.254665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.254665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.254665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.254665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.254665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.254665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.254665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:42.254665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:42.254665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:42.254665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.254665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c99624c7-a5ce-4940-ae6a-d4c43c353efd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Returning 404 to user: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:42.254665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c99624c7-a5ce-4940-ae6a-d4c43c353efd tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65" status: 404 len: 109 microversion: 2.1 time: 0.075530 Sep 08 14:29:42.255680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1394/2494] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 => generated 109 bytes in 77 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:29:42.255680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a77cfc1f-f9f7-4034-a80d-530950d70716 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '.version_select at 0x7fce021c3040>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.266760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-a77cfc1f-f9f7-4034-a80d-530950d70716 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] HTTP exception thrown: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a77cfc1f-f9f7-4034-a80d-530950d70716 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Request method failure captured: Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013 HTTP/1.1 Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPpgseYqW3vfBhChnnflETRt0GDtjL3VmhITeNLuRSALZefRpZ4LlXgFeo5cygwUtHgGuuBrrkR39Uo-ZWGIlwSJVssX0SoJbjjdLhYxjZOutCiSryd_0UQo0F3jz05Y7DAMd5jNO7kUKOE8WKEghkWPfZegSJZuLFAO1D7uF_weRo Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Openstack-Nova-Api-Version: 2.20 Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:29:42.268494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:29:42.270701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:42.270701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:42.270701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:29:42.270701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:42.270701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:29:42.270701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:29:42.270701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:29:42.270701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:42.270701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:29:42.270701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: Sep 08 14:29:42.270701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:42.270701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'tempest-keypair-920229013'} Sep 08 14:29:42.270701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:29:42.271917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:42.273916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 160, in delete Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._delete(req, id, user_id=user_id) Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:42.275583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.277132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a77cfc1f-f9f7-4034-a80d-530950d70716 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Returning 404 to user: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:42.277132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a77cfc1f-f9f7-4034-a80d-530950d70716 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013" status: 404 len: 131 microversion: 2.20 time: 0.019317 Sep 08 14:29:42.277132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1395/2495] 10.222.0.25 () {60 vars in 1259 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013 => generated 131 bytes in 20 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 08 14:29:42.277132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-65030da7-47cc-426d-b0d4-b9785706d986 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.277132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-65030da7-47cc-426d-b0d4-b9785706d986 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:42.282950 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-65030da7-47cc-426d-b0d4-b9785706d986 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.283438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-65030da7-47cc-426d-b0d4-b9785706d986 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.283877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-65030da7-47cc-426d-b0d4-b9785706d986 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.284574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-65030da7-47cc-426d-b0d4-b9785706d986 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.285051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-65030da7-47cc-426d-b0d4-b9785706d986 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.285454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-65030da7-47cc-426d-b0d4-b9785706d986 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.291239 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.compute.api [None req-0bb41242-5ffe-4258-9c92-25254becfc25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Instance is already in deleting state, ignoring this request Sep 08 14:29:42.292060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0bb41242-5ffe-4258-9c92-25254becfc25 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 204 len: 0 microversion: 2.1 time: 0.085249 Sep 08 14:29:42.292727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1101/2496] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 0 bytes in 86 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:29:42.297194 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ce28a698-9a65-46e3-8aeb-63b337db4f82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '.version_select at 0x7fce0589e310>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.310571 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-65030da7-47cc-426d-b0d4-b9785706d986 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:42.396131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ce28a698-9a65-46e3-8aeb-63b337db4f82 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-720914496" status: 202 len: 0 microversion: 2.1 time: 0.102343 Sep 08 14:29:42.397051 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1102/2497] 10.222.0.25 () {58 vars in 1260 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-720914496 => generated 0 bytes in 104 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:29:42.401907 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d3a42f93-02e4-41a6-9512-76a465ca87c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.402425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d3a42f93-02e4-41a6-9512-76a465ca87c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:42.410882 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d3a42f93-02e4-41a6-9512-76a465ca87c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.411413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d3a42f93-02e4-41a6-9512-76a465ca87c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.411879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d3a42f93-02e4-41a6-9512-76a465ca87c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.413653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d3a42f93-02e4-41a6-9512-76a465ca87c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.414169 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d3a42f93-02e4-41a6-9512-76a465ca87c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.414634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d3a42f93-02e4-41a6-9512-76a465ca87c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.441359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d3a42f93-02e4-41a6-9512-76a465ca87c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:42.491314 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.compute.api [None req-d3a42f93-02e4-41a6-9512-76a465ca87c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Instance is already in deleting state, ignoring this request Sep 08 14:29:42.491738 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d3a42f93-02e4-41a6-9512-76a465ca87c7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 204 len: 0 microversion: 2.1 time: 0.093575 Sep 08 14:29:42.492296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1103/2498] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:29:42.497257 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6b319d9b-4f3d-48e9-af85-e5c17e5676a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '.version_select at 0x7fce059831f0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.508884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-6b319d9b-4f3d-48e9-af85-e5c17e5676a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6b319d9b-4f3d-48e9-af85-e5c17e5676a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 HTTP/1.1 Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:29:42.510796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:29:42.513495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:42.513495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:42.513495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:29:42.513495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:42.513495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:29:42.513495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:29:42.513495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:29:42.513495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:29:42.513495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:29:42.513495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 14:29:42.513495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:42.513495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'tempest-keypair-997401276'} Sep 08 14:29:42.513495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:29:42.514616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:42.516185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:42.518015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:42.518015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:42.518015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 08 14:29:42.518015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 08 14:29:42.518015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:29:42.518015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:29:42.518015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:42.518015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.518015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.518015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.518015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.518015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.518015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.518015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:42.518015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:42.518015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:42.518015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.519426 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6b319d9b-4f3d-48e9-af85-e5c17e5676a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:42.519426 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6b319d9b-4f3d-48e9-af85-e5c17e5676a8 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276" status: 404 len: 131 microversion: 2.1 time: 0.019283 Sep 08 14:29:42.519426 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1104/2499] 10.222.0.25 () {58 vars in 1218 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 => generated 131 bytes in 21 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:29:42.519426 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c2c56958-cc1a-41af-b01b-118b64c3df87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.519426 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c2c56958-cc1a-41af-b01b-118b64c3df87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:42.676767 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-65030da7-47cc-426d-b0d4-b9785706d986 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 204 len: 0 microversion: 2.1 time: 0.404953 Sep 08 14:29:42.677346 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1396/2500] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:29:42.681328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2c56958-cc1a-41af-b01b-118b64c3df87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.681814 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2c56958-cc1a-41af-b01b-118b64c3df87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.682266 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2c56958-cc1a-41af-b01b-118b64c3df87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.682957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2c56958-cc1a-41af-b01b-118b64c3df87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.683407 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2c56958-cc1a-41af-b01b-118b64c3df87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.683924 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c2c56958-cc1a-41af-b01b-118b64c3df87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.686772 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-dd0e2f3a-8699-414a-99bf-5e1eb4a528d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.687353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-dd0e2f3a-8699-414a-99bf-5e1eb4a528d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:42.693901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dd0e2f3a-8699-414a-99bf-5e1eb4a528d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.694494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dd0e2f3a-8699-414a-99bf-5e1eb4a528d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.695028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dd0e2f3a-8699-414a-99bf-5e1eb4a528d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.695836 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dd0e2f3a-8699-414a-99bf-5e1eb4a528d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.696358 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dd0e2f3a-8699-414a-99bf-5e1eb4a528d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.696853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-dd0e2f3a-8699-414a-99bf-5e1eb4a528d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.725454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-dd0e2f3a-8699-414a-99bf-5e1eb4a528d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:42.745730 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-c2c56958-cc1a-41af-b01b-118b64c3df87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] HTTP exception thrown: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c2c56958-cc1a-41af-b01b-118b64c3df87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Request method failure captured: Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 HTTP/1.1 Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfSsEI3OQCjaT8uW9n_C9Vd1UPOl7hGg4pp7-4nDKa37scg2VH8UGybcHZqyDn5ybTSjRS0vhV5EoGnyfNs_A0uLCWs9ZtiXuHPkrFMPvuwPxHSIVoo5s0Kpv1P1uWF7oFwsg1AH73mWHbrcQqseds7NX0AeiotGfcerp1SS6-JaE8A Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:29:42.748968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:29:42.750952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_179292c42f8e4041adc935c80dba0f70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:42.750952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:42.750952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:29:42.750952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:42.750952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:29:42.750952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:29:42.750952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:29:42.750952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 1f10a95e49744a6dbb73e54e00d81add Sep 08 14:29:42.750952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:29:42.750952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:29:42.750952 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:42.751881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'server_id': 'd7ae43e4-11c8-417c-be73-4f9fb5c42f2c', 'id': '9a4342d7-3883-4385-bcd2-b4070743be65'} Sep 08 14:29:42.751881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:42.751881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.751881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:29:42.751881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:29:42.751881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 08 14:29:42.751881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 08 14:29:42.751881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 08 14:29:42.751881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 08 14:29:42.751881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 08 14:29:42.751881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 08 14:29:42.751881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 08 14:29:42.751881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 08 14:29:42.751881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:29:42.751881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:29:42.751881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:29:42.751881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:29:42.751881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. (HTTP 404) (Request-ID: req-727817c0-eaf0-42c6-9b76-dee17425d4c7) Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 547, in delete Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi volume = self.volume_api.get(context, volume_id) Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 08 14:29:42.754124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 549, in delete Sep 08 14:29:42.760000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:29:42.762062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:42.762062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.762062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.762062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.762062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.762062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.762062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.762062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:42.762062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:42.762062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:42.762062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.762062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c2c56958-cc1a-41af-b01b-118b64c3df87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Returning 404 to user: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:42.762062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c2c56958-cc1a-41af-b01b-118b64c3df87 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65" status: 404 len: 109 microversion: 2.1 time: 0.235961 Sep 08 14:29:42.764541 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1105/2501] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 => generated 109 bytes in 237 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:29:42.764541 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-350e1283-bc01-4eb8-bc65-c1834d396322 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '.version_select at 0x7fce0597dc10>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.764826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=5,nova_cell1:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:29:42.772105 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-350e1283-bc01-4eb8-bc65-c1834d396322 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] HTTP exception thrown: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-350e1283-bc01-4eb8-bc65-c1834d396322 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Request method failure captured: Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013 HTTP/1.1 Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPpgseYqW3vfBhChnnflETRt0GDtjL3VmhITeNLuRSALZefRpZ4LlXgFeo5cygwUtHgGuuBrrkR39Uo-ZWGIlwSJVssX0SoJbjjdLhYxjZOutCiSryd_0UQo0F3jz05Y7DAMd5jNO7kUKOE8WKEghkWPfZegSJZuLFAO1D7uF_weRo Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Openstack-Nova-Api-Version: 2.20 Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:29:42.774186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:29:42.776023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:42.776023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:42.776023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:29:42.776023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:42.776023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:29:42.776023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:29:42.776023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:29:42.776023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:42.776023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:29:42.776023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 14:29:42.776023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:42.776023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'tempest-keypair-920229013'} Sep 08 14:29:42.776023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:29:42.777676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:42.779186 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 160, in delete Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req, id, user_id=user_id) Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:42.780656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.782416 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-350e1283-bc01-4eb8-bc65-c1834d396322 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Returning 404 to user: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:42.782416 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-350e1283-bc01-4eb8-bc65-c1834d396322 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013" status: 404 len: 131 microversion: 2.20 time: 0.023616 Sep 08 14:29:42.782416 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1106/2502] 10.222.0.25 () {60 vars in 1259 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013 => generated 131 bytes in 25 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 08 14:29:42.782416 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fa989266-849f-43fe-bcdc-3a789c0918f6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.782416 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fa989266-849f-43fe-bcdc-3a789c0918f6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:42.787507 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa989266-849f-43fe-bcdc-3a789c0918f6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.788062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa989266-849f-43fe-bcdc-3a789c0918f6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.788582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa989266-849f-43fe-bcdc-3a789c0918f6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.789440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa989266-849f-43fe-bcdc-3a789c0918f6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.790486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa989266-849f-43fe-bcdc-3a789c0918f6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.790486 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fa989266-849f-43fe-bcdc-3a789c0918f6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.795322 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=93,nova_cell1:UPDATE=6,nova_cell1:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:29:42.817697 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fa989266-849f-43fe-bcdc-3a789c0918f6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:42.844658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.compute.api [None req-dd0e2f3a-8699-414a-99bf-5e1eb4a528d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Instance is already in deleting state, ignoring this request Sep 08 14:29:42.845505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-dd0e2f3a-8699-414a-99bf-5e1eb4a528d4 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 204 len: 0 microversion: 2.1 time: 0.167087 Sep 08 14:29:42.846460 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1397/2503] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:29:42.851586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-389ea9f4-32ea-4431-a201-fd8f51726019 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '.version_select at 0x7fce01c99040>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.864064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-389ea9f4-32ea-4431-a201-fd8f51726019 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] HTTP exception thrown: Keypair tempest-TestVolumeBootPattern-720914496 not found for user 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-389ea9f4-32ea-4431-a201-fd8f51726019 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Request method failure captured: Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-720914496 HTTP/1.1 Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfR-noSyVFO_CwfvnHm1tKMROORj39_aeYmVL57cJH5P0CKIBxGmupxXskOhl3govhUXvkZDwtrF-q1QWbqRhaHpW2fGrN3ado0HHqqQh_E0h7uwLaTdYOzkjbpdS4KTgY9PGN8tIW4uf_fu6nXCzaHGAE-HkeES6UOy8csw-kGPtKA Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:29:42.865893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:29:42.867729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:42.867729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-TestVolumeBootPattern-588128932 Sep 08 14:29:42.867729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:29:42.867729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:29:42.867729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:29:42.867729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:29:42.867729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:29:42.867729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:29:42.867729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:29:42.867729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: Sep 08 14:29:42.867729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Keypair tempest-TestVolumeBootPattern-720914496 not found for user 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:29:42.867729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'tempest-TestVolumeBootPattern-720914496'} Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:29:42.868992 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-TestVolumeBootPattern-720914496 not found for user 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:29:42.870522 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:42.872784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:42.872784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:42.872784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:42.872784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 08 14:29:42.872784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 08 14:29:42.872784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:29:42.872784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:29:42.872784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-TestVolumeBootPattern-720914496 not found for user 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:29:42.872784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.872784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.872784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.872784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.872784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.872784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.872784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:42.872784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:42.872784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-TestVolumeBootPattern-720914496 not found for user 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:29:42.872784 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.874923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-389ea9f4-32ea-4431-a201-fd8f51726019 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Returning 404 to user: Keypair tempest-TestVolumeBootPattern-720914496 not found for user 772205d93ce047e9ad4ddefadec300f4 {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:42.874923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-389ea9f4-32ea-4431-a201-fd8f51726019 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-720914496" status: 404 len: 145 microversion: 2.1 time: 0.019941 Sep 08 14:29:42.874923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1398/2504] 10.222.0.25 () {58 vars in 1260 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-720914496 => generated 145 bytes in 21 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:29:42.874923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1d0d538b-6978-4b38-9c4e-50cd086dc98f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.874923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1d0d538b-6978-4b38-9c4e-50cd086dc98f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:42.880960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d0d538b-6978-4b38-9c4e-50cd086dc98f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.881618 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d0d538b-6978-4b38-9c4e-50cd086dc98f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.882049 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d0d538b-6978-4b38-9c4e-50cd086dc98f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.882765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d0d538b-6978-4b38-9c4e-50cd086dc98f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.883214 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d0d538b-6978-4b38-9c4e-50cd086dc98f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.883634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1d0d538b-6978-4b38-9c4e-50cd086dc98f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.910423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1d0d538b-6978-4b38-9c4e-50cd086dc98f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:42.942520 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.compute.api [None req-fa989266-849f-43fe-bcdc-3a789c0918f6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Instance is already in deleting state, ignoring this request Sep 08 14:29:42.943921 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fa989266-849f-43fe-bcdc-3a789c0918f6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 204 len: 0 microversion: 2.1 time: 0.166663 Sep 08 14:29:42.945097 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1107/2505] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:29:42.954366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d1ba73de-f529-49ab-b318-3d85c0174053 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '.version_select at 0x7fce05992f70>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.970671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-d1ba73de-f529-49ab-b318-3d85c0174053 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d1ba73de-f529-49ab-b318-3d85c0174053 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 HTTP/1.1 Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:29:42.973199 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:29:42.974959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:42.974959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:42.974959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:29:42.974959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:42.974959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:29:42.974959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:29:42.974959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:29:42.974959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:29:42.974959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:29:42.974959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 14:29:42.974959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:42.974959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'tempest-keypair-997401276'} Sep 08 14:29:42.974959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:29:42.976018 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:42.977727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:42.979348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:42.979348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:42.979348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 08 14:29:42.979348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 08 14:29:42.979348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:29:42.979348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:29:42.979348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:42.979348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.979348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:42.979348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.979348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:42.979348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:42.979348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:42.979348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:42.979348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:42.979348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:42.979348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:42.980766 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d1ba73de-f529-49ab-b318-3d85c0174053 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:42.980766 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d1ba73de-f529-49ab-b318-3d85c0174053 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276" status: 404 len: 131 microversion: 2.1 time: 0.027719 Sep 08 14:29:42.980766 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1108/2506] 10.222.0.25 () {58 vars in 1218 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 => generated 131 bytes in 29 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:29:42.980766 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0d9ca2c5-c85d-4e1c-98d3-0bc1ab7aebc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:42.980766 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0d9ca2c5-c85d-4e1c-98d3-0bc1ab7aebc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:42.986878 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0d9ca2c5-c85d-4e1c-98d3-0bc1ab7aebc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.987359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0d9ca2c5-c85d-4e1c-98d3-0bc1ab7aebc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.987799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0d9ca2c5-c85d-4e1c-98d3-0bc1ab7aebc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:42.988628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0d9ca2c5-c85d-4e1c-98d3-0bc1ab7aebc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:42.989085 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0d9ca2c5-c85d-4e1c-98d3-0bc1ab7aebc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:42.989500 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0d9ca2c5-c85d-4e1c-98d3-0bc1ab7aebc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:43.025569 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.compute.api [None req-1d0d538b-6978-4b38-9c4e-50cd086dc98f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Instance is already in deleting state, ignoring this request Sep 08 14:29:43.026441 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1d0d538b-6978-4b38-9c4e-50cd086dc98f tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 204 len: 0 microversion: 2.1 time: 0.157627 Sep 08 14:29:43.027047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1399/2507] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:29:43.031711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2b1372ec-2de2-4341-9fe9-5744007d893b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:43.032206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2b1372ec-2de2-4341-9fe9-5744007d893b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:43.038076 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b1372ec-2de2-4341-9fe9-5744007d893b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:43.038597 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b1372ec-2de2-4341-9fe9-5744007d893b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:43.039039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b1372ec-2de2-4341-9fe9-5744007d893b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:43.039737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b1372ec-2de2-4341-9fe9-5744007d893b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:43.040203 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b1372ec-2de2-4341-9fe9-5744007d893b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:43.040625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2b1372ec-2de2-4341-9fe9-5744007d893b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:43.041265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-0d9ca2c5-c85d-4e1c-98d3-0bc1ab7aebc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] HTTP exception thrown: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0d9ca2c5-c85d-4e1c-98d3-0bc1ab7aebc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Request method failure captured: Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 HTTP/1.1 Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfSsEI3OQCjaT8uW9n_C9Vd1UPOl7hGg4pp7-4nDKa37scg2VH8UGybcHZqyDn5ybTSjRS0vhV5EoGnyfNs_A0uLCWs9ZtiXuHPkrFMPvuwPxHSIVoo5s0Kpv1P1uWF7oFwsg1AH73mWHbrcQqseds7NX0AeiotGfcerp1SS6-JaE8A Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:29:43.043325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:29:43.045580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_179292c42f8e4041adc935c80dba0f70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:43.045580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:43.045580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:29:43.045580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:43.045580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:29:43.045580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:29:43.045580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:29:43.045580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 1f10a95e49744a6dbb73e54e00d81add Sep 08 14:29:43.045580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:29:43.045580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:29:43.045580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:43.046613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'server_id': 'd7ae43e4-11c8-417c-be73-4f9fb5c42f2c', 'id': '9a4342d7-3883-4385-bcd2-b4070743be65'} Sep 08 14:29:43.046613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:43.046613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.046613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:29:43.046613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:29:43.046613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 08 14:29:43.046613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 08 14:29:43.046613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 08 14:29:43.046613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 08 14:29:43.046613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 08 14:29:43.046613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 08 14:29:43.046613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 08 14:29:43.046613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 08 14:29:43.046613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:29:43.046613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:29:43.046613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:29:43.046613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:29:43.046613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. (HTTP 404) (Request-ID: req-28cf3cef-4b9c-4b15-a740-7575c19f9032) Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 547, in delete Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi volume = self.volume_api.get(context, volume_id) Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 08 14:29:43.048276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 549, in delete Sep 08 14:29:43.049890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:29:43.051808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:43.051808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.051808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:43.051808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.051808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.051808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:43.051808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:43.051808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:43.051808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:43.051808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. Sep 08 14:29:43.051808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.051808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0d9ca2c5-c85d-4e1c-98d3-0bc1ab7aebc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Returning 404 to user: Volume 9a4342d7-3883-4385-bcd2-b4070743be65 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:43.051808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0d9ca2c5-c85d-4e1c-98d3-0bc1ab7aebc9 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65" status: 404 len: 109 microversion: 2.1 time: 0.069854 Sep 08 14:29:43.052967 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1109/2508] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:29:42 2022] DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c/os-volume_attachments/9a4342d7-3883-4385-bcd2-b4070743be65 => generated 109 bytes in 71 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:29:43.052967 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a57d3953-f76c-4a73-92bf-32b4dfc44223 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '.version_select at 0x7fce059925e0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:43.063064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-a57d3953-f76c-4a73-92bf-32b4dfc44223 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] HTTP exception thrown: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a57d3953-f76c-4a73-92bf-32b4dfc44223 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Request method failure captured: Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013 HTTP/1.1 Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPpgseYqW3vfBhChnnflETRt0GDtjL3VmhITeNLuRSALZefRpZ4LlXgFeo5cygwUtHgGuuBrrkR39Uo-ZWGIlwSJVssX0SoJbjjdLhYxjZOutCiSryd_0UQo0F3jz05Y7DAMd5jNO7kUKOE8WKEghkWPfZegSJZuLFAO1D7uF_weRo Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Openstack-Nova-Api-Version: 2.20 Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:29:43.064466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:29:43.066527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:43.066527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:43.066527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:29:43.066527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:43.066527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:29:43.066527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:29:43.066527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:29:43.066527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:43.066527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:29:43.066527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 14:29:43.066527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:43.066527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'tempest-keypair-920229013'} Sep 08 14:29:43.066527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:29:43.067737 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:43.069456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 160, in delete Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req, id, user_id=user_id) Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:43.072084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.073906 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a57d3953-f76c-4a73-92bf-32b4dfc44223 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Returning 404 to user: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:43.073906 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a57d3953-f76c-4a73-92bf-32b4dfc44223 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013" status: 404 len: 131 microversion: 2.20 time: 0.018928 Sep 08 14:29:43.073906 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1110/2509] 10.222.0.25 () {60 vars in 1259 bytes} [Thu Sep 8 14:29:43 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013 => generated 131 bytes in 20 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 08 14:29:43.074209 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2b1372ec-2de2-4341-9fe9-5744007d893b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:43.074334 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-29f535f8-5b4f-4881-ad40-dcbb7cf17ce1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:43.074334 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-29f535f8-5b4f-4881-ad40-dcbb7cf17ce1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:43.078456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29f535f8-5b4f-4881-ad40-dcbb7cf17ce1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:43.078920 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29f535f8-5b4f-4881-ad40-dcbb7cf17ce1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:43.079343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29f535f8-5b4f-4881-ad40-dcbb7cf17ce1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:43.080047 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29f535f8-5b4f-4881-ad40-dcbb7cf17ce1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:43.080474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29f535f8-5b4f-4881-ad40-dcbb7cf17ce1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:43.080891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29f535f8-5b4f-4881-ad40-dcbb7cf17ce1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:43.108546 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-29f535f8-5b4f-4881-ad40-dcbb7cf17ce1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:43.119584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.compute.api [None req-2b1372ec-2de2-4341-9fe9-5744007d893b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Instance is already in deleting state, ignoring this request Sep 08 14:29:43.120402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2b1372ec-2de2-4341-9fe9-5744007d893b tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 204 len: 0 microversion: 2.1 time: 0.092457 Sep 08 14:29:43.120963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1400/2510] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:43 2022] DELETE /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 0 bytes in 93 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:29:43.125407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-303e9d31-fddd-4ab2-a39f-1e76b27ab396 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '.version_select at 0x7fce058409d0>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:43.136815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-303e9d31-fddd-4ab2-a39f-1e76b27ab396 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] HTTP exception thrown: Keypair tempest-TestVolumeBootPattern-720914496 not found for user 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-303e9d31-fddd-4ab2-a39f-1e76b27ab396 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Request method failure captured: Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-720914496 HTTP/1.1 Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfR-noSyVFO_CwfvnHm1tKMROORj39_aeYmVL57cJH5P0CKIBxGmupxXskOhl3govhUXvkZDwtrF-q1QWbqRhaHpW2fGrN3ado0HHqqQh_E0h7uwLaTdYOzkjbpdS4KTgY9PGN8tIW4uf_fu6nXCzaHGAE-HkeES6UOy8csw-kGPtKA Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:29:43.138925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:29:43.140528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:43.140528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-TestVolumeBootPattern-588128932 Sep 08 14:29:43.140528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:29:43.140528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:29:43.140528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:29:43.140528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:29:43.140528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:29:43.140528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:29:43.140528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:29:43.140528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: Sep 08 14:29:43.140528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Keypair tempest-TestVolumeBootPattern-720914496 not found for user 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:29:43.140528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'tempest-TestVolumeBootPattern-720914496'} Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:29:43.141472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-TestVolumeBootPattern-720914496 not found for user 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:29:43.143345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:43.145152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:43.145152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:43.145152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:43.145152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 08 14:29:43.145152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 08 14:29:43.145152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:29:43.145152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:29:43.145152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-TestVolumeBootPattern-720914496 not found for user 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:29:43.145152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.145152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:43.145152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.145152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.145152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:43.145152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:43.145152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:43.145152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:43.145152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-TestVolumeBootPattern-720914496 not found for user 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:29:43.145152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.147099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-303e9d31-fddd-4ab2-a39f-1e76b27ab396 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Returning 404 to user: Keypair tempest-TestVolumeBootPattern-720914496 not found for user 772205d93ce047e9ad4ddefadec300f4 {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:43.147099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-303e9d31-fddd-4ab2-a39f-1e76b27ab396 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-720914496" status: 404 len: 145 microversion: 2.1 time: 0.019021 Sep 08 14:29:43.147099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1401/2511] 10.222.0.25 () {58 vars in 1260 bytes} [Thu Sep 8 14:29:43 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-720914496 => generated 145 bytes in 20 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:29:43.147099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1763590e-800a-4952-9388-77760b506d1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:43.147099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1763590e-800a-4952-9388-77760b506d1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:43.152780 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1763590e-800a-4952-9388-77760b506d1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:43.153806 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1763590e-800a-4952-9388-77760b506d1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:43.154272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1763590e-800a-4952-9388-77760b506d1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:43.154840 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.compute.api [None req-29f535f8-5b4f-4881-ad40-dcbb7cf17ce1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Instance is already in deleting state, ignoring this request Sep 08 14:29:43.154955 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1763590e-800a-4952-9388-77760b506d1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:43.155400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1763590e-800a-4952-9388-77760b506d1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:43.155658 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-29f535f8-5b4f-4881-ad40-dcbb7cf17ce1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 204 len: 0 microversion: 2.1 time: 0.088212 Sep 08 14:29:43.155853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1763590e-800a-4952-9388-77760b506d1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:43.156202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1111/2512] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:43 2022] DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:29:43.160641 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-67adb9ba-1fbb-4edb-8b92-ed7ba1748dd1 req-3886e558-3d45-440d-bc71-5362e73c48d5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "19db4aa2-f574-43fa-bbf8-c3042b243152", "name": "network-vif-deleted", "tag": "2ebf3a50-af15-4ecc-820b-87ad0a9ff652"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:29:43.170092 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-67adb9ba-1fbb-4edb-8b92-ed7ba1748dd1 req-3886e558-3d45-440d-bc71-5362e73c48d5 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:43.170585 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-67adb9ba-1fbb-4edb-8b92-ed7ba1748dd1 req-3886e558-3d45-440d-bc71-5362e73c48d5 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:43.171006 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-67adb9ba-1fbb-4edb-8b92-ed7ba1748dd1 req-3886e558-3d45-440d-bc71-5362e73c48d5 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:43.181649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1763590e-800a-4952-9388-77760b506d1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:43.182372 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-67adb9ba-1fbb-4edb-8b92-ed7ba1748dd1 req-3886e558-3d45-440d-bc71-5362e73c48d5 service nova] Creating event network-vif-deleted:2ebf3a50-af15-4ecc-820b-87ad0a9ff652 for instance 19db4aa2-f574-43fa-bbf8-c3042b243152 on hv-cinder-85646 Sep 08 14:29:43.189013 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-67adb9ba-1fbb-4edb-8b92-ed7ba1748dd1 req-3886e558-3d45-440d-bc71-5362e73c48d5 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030050 Sep 08 14:29:43.189013 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1112/2513] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:29:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 31 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:29:43.200585 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-198ba7c3-3a9d-474d-9c91-9eaf6769434d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '.version_select at 0x7fce059caaf0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:43.216991 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-198ba7c3-3a9d-474d-9c91-9eaf6769434d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-198ba7c3-3a9d-474d-9c91-9eaf6769434d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 HTTP/1.1 Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:29:43.220049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:29:43.221869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:43.221869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:43.221869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:29:43.221869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:29:43.221869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:29:43.221869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:29:43.221869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:29:43.221869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:29:43.221869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:29:43.221869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 14:29:43.221869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:43.221869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'tempest-keypair-997401276'} Sep 08 14:29:43.221869 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:29:43.222957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:43.224467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:43.226012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:43.226012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:43.226012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 08 14:29:43.226012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 08 14:29:43.226012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:29:43.226012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:29:43.226012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:43.226012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.226012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:43.226012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.226012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.226012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:43.226012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:43.226012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:43.226012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:43.226012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc Sep 08 14:29:43.226012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.228063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-198ba7c3-3a9d-474d-9c91-9eaf6769434d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Keypair tempest-keypair-997401276 not found for user eca89f435299415c8448931e73f5b0fc {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:43.229095 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-198ba7c3-3a9d-474d-9c91-9eaf6769434d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276" status: 404 len: 131 microversion: 2.1 time: 0.040039 Sep 08 14:29:43.230109 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1113/2514] 10.222.0.25 () {58 vars in 1218 bytes} [Thu Sep 8 14:29:43 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-997401276 => generated 131 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:29:43.241239 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a28b656c-385f-4c4d-86db-870907b5dbf5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:43.241239 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a28b656c-385f-4c4d-86db-870907b5dbf5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:43.244010 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a28b656c-385f-4c4d-86db-870907b5dbf5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:43.244489 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a28b656c-385f-4c4d-86db-870907b5dbf5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:43.245149 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a28b656c-385f-4c4d-86db-870907b5dbf5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:43.245850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a28b656c-385f-4c4d-86db-870907b5dbf5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:43.246343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a28b656c-385f-4c4d-86db-870907b5dbf5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:43.246867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a28b656c-385f-4c4d-86db-870907b5dbf5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:43.283250 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a28b656c-385f-4c4d-86db-870907b5dbf5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:43.283738 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a28b656c-385f-4c4d-86db-870907b5dbf5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:43.284143 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a28b656c-385f-4c4d-86db-870907b5dbf5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:43.284583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-a28b656c-385f-4c4d-86db-870907b5dbf5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lazy-loading 'fault' on Instance uuid d1062abf-7525-48c5-a9cb-91da3855fd1b {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:29:43.328832 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a28b656c-385f-4c4d-86db-870907b5dbf5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f2026170f4674b63a37534c14689bd10', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5be47c3b2c4c4fc2a1db117433cbcfc4', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:43.337842 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_db.api [None req-1763590e-800a-4952-9388-77760b506d1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Performing DB retry for function nova.db.main.api.instance_update_and_get_original {{(pid=127601) wrapper /usr/local/lib/python3.8/dist-packages/oslo_db/api.py:155}} Sep 08 14:29:43.339719 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a28b656c-385f-4c4d-86db-870907b5dbf5 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 200 len: 1319 microversion: 2.1 time: 0.108458 Sep 08 14:29:43.341006 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1114/2515] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:43 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 1319 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:29:43.345654 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c6589e7d-7f5f-40f6-a092-af53b9a5d93a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '.version_select at 0x7fce05992e50>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:43.362640 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-c6589e7d-7f5f-40f6-a092-af53b9a5d93a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] HTTP exception thrown: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c6589e7d-7f5f-40f6-a092-af53b9a5d93a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Request method failure captured: Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013 HTTP/1.1 Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPpgseYqW3vfBhChnnflETRt0GDtjL3VmhITeNLuRSALZefRpZ4LlXgFeo5cygwUtHgGuuBrrkR39Uo-ZWGIlwSJVssX0SoJbjjdLhYxjZOutCiSryd_0UQo0F3jz05Y7DAMd5jNO7kUKOE8WKEghkWPfZegSJZuLFAO1D7uF_weRo Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Openstack-Nova-Api-Version: 2.20 Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:29:43.364651 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:29:43.366390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:43.366390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:43.366390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:29:43.366390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:29:43.366390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:29:43.366390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:29:43.366390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:29:43.366390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:43.366390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:29:43.366390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 14:29:43.366390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:43.366390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'tempest-keypair-920229013'} Sep 08 14:29:43.366390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:29:43.367351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:43.369256 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 160, in delete Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req, id, user_id=user_id) Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 Sep 08 14:29:43.371535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.373269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c6589e7d-7f5f-40f6-a092-af53b9a5d93a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Returning 404 to user: Keypair tempest-keypair-920229013 not found for user e9925313d1d446fa81a00cb55348dff7 {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:43.373269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c6589e7d-7f5f-40f6-a092-af53b9a5d93a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013" status: 404 len: 131 microversion: 2.20 time: 0.024061 Sep 08 14:29:43.373269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1115/2516] 10.222.0.25 () {60 vars in 1259 bytes} [Thu Sep 8 14:29:43 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-920229013 => generated 131 bytes in 26 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 08 14:29:43.373269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b538cb5f-119d-448f-b96a-e403d4515776 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:43.373269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b538cb5f-119d-448f-b96a-e403d4515776 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:43.379930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b538cb5f-119d-448f-b96a-e403d4515776 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:43.380448 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b538cb5f-119d-448f-b96a-e403d4515776 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:43.380909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b538cb5f-119d-448f-b96a-e403d4515776 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:43.381607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b538cb5f-119d-448f-b96a-e403d4515776 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:43.382063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b538cb5f-119d-448f-b96a-e403d4515776 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:43.382531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b538cb5f-119d-448f-b96a-e403d4515776 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:43.409747 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b538cb5f-119d-448f-b96a-e403d4515776 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:43.461485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.compute.api [None req-b538cb5f-119d-448f-b96a-e403d4515776 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Instance is already in deleting state, ignoring this request Sep 08 14:29:43.461485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b538cb5f-119d-448f-b96a-e403d4515776 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 204 len: 0 microversion: 2.1 time: 0.090307 Sep 08 14:29:43.461485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1116/2517] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:43 2022] DELETE /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:29:43.464125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9c443a9f-1736-4c60-87ee-7515757c1293 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Calling method '.version_select at 0x7fce059cadc0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:43.477765 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-9c443a9f-1736-4c60-87ee-7515757c1293 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] HTTP exception thrown: Keypair tempest-TestVolumeBootPattern-720914496 not found for user 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9c443a9f-1736-4c60-87ee-7515757c1293 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Request method failure captured: Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-720914496 HTTP/1.1 Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfR-noSyVFO_CwfvnHm1tKMROORj39_aeYmVL57cJH5P0CKIBxGmupxXskOhl3govhUXvkZDwtrF-q1QWbqRhaHpW2fGrN3ado0HHqqQh_E0h7uwLaTdYOzkjbpdS4KTgY9PGN8tIW4uf_fu6nXCzaHGAE-HkeES6UOy8csw-kGPtKA Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:29:43.479734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:29:43.481328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/45579d93010d4ed0b80cad3a8b85ea50"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:43.481328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-TestVolumeBootPattern-588128932 Sep 08 14:29:43.481328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 45579d93010d4ed0b80cad3a8b85ea50 Sep 08 14:29:43.481328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-TestVolumeBootPattern-588128932 Sep 08 14:29:43.481328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:29:43.481328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:29:43.481328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:29:43.481328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:29:43.481328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-TestVolumeBootPattern-588128932-project Sep 08 14:29:43.481328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 14:29:43.481328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Keypair tempest-TestVolumeBootPattern-720914496 not found for user 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:29:43.481328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'tempest-TestVolumeBootPattern-720914496'} Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise self.value Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 08 14:29:43.482928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-TestVolumeBootPattern-720914496 not found for user 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:29:43.484504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 08 14:29:43.485876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 08 14:29:43.485876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:43.485876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:43.485876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 08 14:29:43.485876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 08 14:29:43.485876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 08 14:29:43.485876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 08 14:29:43.485876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-TestVolumeBootPattern-720914496 not found for user 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:29:43.485876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.485876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:43.485876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.485876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.485876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:43.485876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:43.485876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:43.485876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:43.485876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-TestVolumeBootPattern-720914496 not found for user 772205d93ce047e9ad4ddefadec300f4 Sep 08 14:29:43.485876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.488462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9c443a9f-1736-4c60-87ee-7515757c1293 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] Returning 404 to user: Keypair tempest-TestVolumeBootPattern-720914496 not found for user 772205d93ce047e9ad4ddefadec300f4 {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:43.488462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9c443a9f-1736-4c60-87ee-7515757c1293 tempest-TestVolumeBootPattern-588128932 tempest-TestVolumeBootPattern-588128932-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-720914496" status: 404 len: 145 microversion: 2.1 time: 0.021203 Sep 08 14:29:43.488462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1117/2518] 10.222.0.25 () {58 vars in 1260 bytes} [Thu Sep 8 14:29:43 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-720914496 => generated 145 bytes in 22 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:29:43.493923 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d1295f85-7765-48dd-b687-dd3a7441befd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:43.496816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d1295f85-7765-48dd-b687-dd3a7441befd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:43.503065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d1295f85-7765-48dd-b687-dd3a7441befd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:43.503825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d1295f85-7765-48dd-b687-dd3a7441befd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:43.504468 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d1295f85-7765-48dd-b687-dd3a7441befd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:43.506294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d1295f85-7765-48dd-b687-dd3a7441befd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:43.507819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d1295f85-7765-48dd-b687-dd3a7441befd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:43.508700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d1295f85-7765-48dd-b687-dd3a7441befd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:43.535777 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-d1295f85-7765-48dd-b687-dd3a7441befd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] HTTP exception thrown: Instance 19db4aa2-f574-43fa-bbf8-c3042b243152 could not be found. Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d1295f85-7765-48dd-b687-dd3a7441befd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Request method failure captured: Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 HTTP/1.1 Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPpGZzMisIeN0rinm76oerWeI1e2n1ZCDvu1_QxGMaqly5J66d47JWYR-nVcthn5vbK6XFUxRx7ULj4gJs_JU089dQK1DqQD0zha4-7jxb_89w0QMqZgtACCW34zdnpCyIfB12dgyAWUWv38AgePo6JrTClIfdFoSLdR5OVWnj2kmY Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 1346d4105dce4c6e928fcb11cde56854 Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-188566618 Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:29:43.539673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:29:43.541588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:43.541588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-ServerRescueNegativeTestJSON-188566618 Sep 08 14:29:43.541588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 1346d4105dce4c6e928fcb11cde56854 Sep 08 14:29:43.541588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-188566618 Sep 08 14:29:43.541588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-ServerRescueNegativeTestJSON-188566618-project Sep 08 14:29:43.541588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:29:43.541588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:29:43.541588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 792808dcbf894aa98c671cf43ac21b8b Sep 08 14:29:43.541588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-ServerRescueNegativeTestJSON-188566618-project Sep 08 14:29:43.541588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:29:43.541588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 19db4aa2-f574-43fa-bbf8-c3042b243152 could not be found. Sep 08 14:29:43.541588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '19db4aa2-f574-43fa-bbf8-c3042b243152'} Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 19db4aa2-f574-43fa-bbf8-c3042b243152 could not be found. Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:43.542861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 19db4aa2-f574-43fa-bbf8-c3042b243152 could not be found. Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 19db4aa2-f574-43fa-bbf8-c3042b243152 could not be found. Sep 08 14:29:43.544653 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:43.547427 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d1295f85-7765-48dd-b687-dd3a7441befd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Returning 404 to user: Instance 19db4aa2-f574-43fa-bbf8-c3042b243152 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:43.548577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d1295f85-7765-48dd-b687-dd3a7441befd tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 404 len: 111 microversion: 2.1 time: 0.065042 Sep 08 14:29:43.549631 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1118/2519] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:43 2022] DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 111 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:29:43.556191 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-86980cb9-0e1e-40af-bfd7-5770a257069b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:43.557300 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-86980cb9-0e1e-40af-bfd7-5770a257069b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:43.566911 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-86980cb9-0e1e-40af-bfd7-5770a257069b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:43.571276 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-86980cb9-0e1e-40af-bfd7-5770a257069b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:43.572880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-86980cb9-0e1e-40af-bfd7-5770a257069b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:43.576265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-86980cb9-0e1e-40af-bfd7-5770a257069b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:43.578627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-86980cb9-0e1e-40af-bfd7-5770a257069b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:43.579480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-86980cb9-0e1e-40af-bfd7-5770a257069b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:43.629150 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-86980cb9-0e1e-40af-bfd7-5770a257069b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:43.914734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-86980cb9-0e1e-40af-bfd7-5770a257069b tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 204 len: 0 microversion: 2.1 time: 0.363539 Sep 08 14:29:43.915403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1119/2520] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:43 2022] DELETE /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 0 bytes in 365 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:29:43.921127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8f5d405a-722b-4dda-b2ce-efd0216a2f30 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:43.921886 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8f5d405a-722b-4dda-b2ce-efd0216a2f30 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:43.928521 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f5d405a-722b-4dda-b2ce-efd0216a2f30 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:43.929477 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f5d405a-722b-4dda-b2ce-efd0216a2f30 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:43.930100 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f5d405a-722b-4dda-b2ce-efd0216a2f30 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:43.931077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f5d405a-722b-4dda-b2ce-efd0216a2f30 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:43.931529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f5d405a-722b-4dda-b2ce-efd0216a2f30 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:43.931951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f5d405a-722b-4dda-b2ce-efd0216a2f30 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:44.163214 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8f5d405a-722b-4dda-b2ce-efd0216a2f30 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:44.185662 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8f5d405a-722b-4dda-b2ce-efd0216a2f30 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1513 microversion: 2.1 time: 0.268614 Sep 08 14:29:44.186179 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1120/2521] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:43 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1513 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:29:44.191110 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-82474e6a-b101-422d-b0be-c5153e024dcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:44.198283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-82474e6a-b101-422d-b0be-c5153e024dcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 19db4aa2-f574-43fa-bbf8-c3042b243152] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:44.198283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-82474e6a-b101-422d-b0be-c5153e024dcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:44.198283 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-82474e6a-b101-422d-b0be-c5153e024dcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:44.198614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-82474e6a-b101-422d-b0be-c5153e024dcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:44.199308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-82474e6a-b101-422d-b0be-c5153e024dcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:44.200549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-82474e6a-b101-422d-b0be-c5153e024dcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:44.200975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-82474e6a-b101-422d-b0be-c5153e024dcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:44.215619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-82474e6a-b101-422d-b0be-c5153e024dcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] HTTP exception thrown: Instance 19db4aa2-f574-43fa-bbf8-c3042b243152 could not be found. Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-82474e6a-b101-422d-b0be-c5153e024dcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Request method failure captured: Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 HTTP/1.1 Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPpGZzMisIeN0rinm76oerWeI1e2n1ZCDvu1_QxGMaqly5J66d47JWYR-nVcthn5vbK6XFUxRx7ULj4gJs_JU089dQK1DqQD0zha4-7jxb_89w0QMqZgtACCW34zdnpCyIfB12dgyAWUWv38AgePo6JrTClIfdFoSLdR5OVWnj2kmY Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 1346d4105dce4c6e928fcb11cde56854 Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-188566618 Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:29:44.217419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:29:44.219296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:44.219296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-ServerRescueNegativeTestJSON-188566618 Sep 08 14:29:44.219296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 1346d4105dce4c6e928fcb11cde56854 Sep 08 14:29:44.219296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-188566618 Sep 08 14:29:44.219296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-ServerRescueNegativeTestJSON-188566618-project Sep 08 14:29:44.219296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:29:44.219296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:29:44.219296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 792808dcbf894aa98c671cf43ac21b8b Sep 08 14:29:44.219296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-ServerRescueNegativeTestJSON-188566618-project Sep 08 14:29:44.219296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:29:44.219296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 19db4aa2-f574-43fa-bbf8-c3042b243152 could not be found. Sep 08 14:29:44.219296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '19db4aa2-f574-43fa-bbf8-c3042b243152'} Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 19db4aa2-f574-43fa-bbf8-c3042b243152 could not be found. Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:44.220202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:44.222476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:44.222476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:29:44.222476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:29:44.222476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:29:44.222476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:29:44.222476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:29:44.222476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:29:44.222476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 19db4aa2-f574-43fa-bbf8-c3042b243152 could not be found. Sep 08 14:29:44.222476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:44.222476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:44.222476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:44.222476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:44.222476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:44.222476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:44.222476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:44.222476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:44.222476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 19db4aa2-f574-43fa-bbf8-c3042b243152 could not be found. Sep 08 14:29:44.222476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:44.224065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-82474e6a-b101-422d-b0be-c5153e024dcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Returning 404 to user: Instance 19db4aa2-f574-43fa-bbf8-c3042b243152 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:44.224065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-82474e6a-b101-422d-b0be-c5153e024dcb tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 404 len: 111 microversion: 2.1 time: 0.031440 Sep 08 14:29:44.224065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1121/2522] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:44 2022] GET /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:29:44.227694 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-813630d4-f252-405d-9a33-70b1be7dd2d5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:44.228308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-813630d4-f252-405d-9a33-70b1be7dd2d5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:44.235223 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-813630d4-f252-405d-9a33-70b1be7dd2d5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:44.235715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-813630d4-f252-405d-9a33-70b1be7dd2d5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:44.236151 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-813630d4-f252-405d-9a33-70b1be7dd2d5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:44.236854 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-813630d4-f252-405d-9a33-70b1be7dd2d5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:44.237306 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-813630d4-f252-405d-9a33-70b1be7dd2d5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:44.237835 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-813630d4-f252-405d-9a33-70b1be7dd2d5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:44.354259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1763590e-800a-4952-9388-77760b506d1e tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152" status: 204 len: 0 microversion: 2.1 time: 1.211782 Sep 08 14:29:44.355306 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1402/2523] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:43 2022] DELETE /compute/v2.1/servers/19db4aa2-f574-43fa-bbf8-c3042b243152 => generated 0 bytes in 1213 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:29:44.361034 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-474e230c-859f-48a9-800f-15a2e5458240 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Action: 'create', calling method: .version_select at 0x7fce021c3ee0>, body: {"keypair": {"name": "tempest-keypair-1145077434", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBB8wv0VKo8Tct6wXhAkTqJBeUsTiBeRL0Nv3UQqRiJO1h5/uAlxLWiXd15UJmst1Xs23Oe5AAq93pCQRTv+Fx3jVx1WiDfrFXF/T+zB7JoCxzTQ+bkVGRSlw+7FG2P+jOQ=="}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:29:44.383491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-474e230c-859f-48a9-800f-15a2e5458240 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Getting quotas for project a4e0fba6cab34c429c73fcfac7ebfd94. Resources: {'key_pairs'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 14:29:44.387055 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-813630d4-f252-405d-9a33-70b1be7dd2d5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:44.387909 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-474e230c-859f-48a9-800f-15a2e5458240 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Getting quotas for user eca89f435299415c8448931e73f5b0fc and project a4e0fba6cab34c429c73fcfac7ebfd94. Resources: {'key_pairs'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 14:29:44.396413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-813630d4-f252-405d-9a33-70b1be7dd2d5 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1552 microversion: 2.1 time: 0.172360 Sep 08 14:29:44.397537 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1122/2524] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:44 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1552 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:29:44.412980 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-134eeffa-132a-401a-ab5c-b583c3272693 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '.version_select at 0x7fce059fa820>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:44.483318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-474e230c-859f-48a9-800f-15a2e5458240 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.126109 Sep 08 14:29:44.484084 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1403/2525] 10.222.0.25 () {60 vars in 1159 bytes} [Thu Sep 8 14:29:44 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:29:44.488685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c5dbed6c-c01c-4ca8-8817-d28af7089861 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:44.489301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c5dbed6c-c01c-4ca8-8817-d28af7089861 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] [instance: d1062abf-7525-48c5-a9cb-91da3855fd1b] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:44.496566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5dbed6c-c01c-4ca8-8817-d28af7089861 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:44.497670 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5dbed6c-c01c-4ca8-8817-d28af7089861 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:44.498115 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5dbed6c-c01c-4ca8-8817-d28af7089861 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:44.499104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5dbed6c-c01c-4ca8-8817-d28af7089861 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:44.501549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5dbed6c-c01c-4ca8-8817-d28af7089861 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:44.502449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c5dbed6c-c01c-4ca8-8817-d28af7089861 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:44.516089 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-134eeffa-132a-401a-ab5c-b583c3272693 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1171611068" status: 202 len: 0 microversion: 2.1 time: 0.117258 Sep 08 14:29:44.516854 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1123/2526] 10.222.0.25 () {58 vars in 1221 bytes} [Thu Sep 8 14:29:44 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1171611068 => generated 0 bytes in 119 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-c5dbed6c-c01c-4ca8-8817-d28af7089861 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] HTTP exception thrown: Instance d1062abf-7525-48c5-a9cb-91da3855fd1b could not be found. Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c5dbed6c-c01c-4ca8-8817-d28af7089861 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Request method failure captured: Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b HTTP/1.1 Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfSAQprctwMMn60-PTTBv8802ViY3wCvJ0L9-BRflVg8jV77kq-O-znQJ3Bdxvg0_CVgAFv7EYiJ2WaKLxpkuwm3jfxMoPZH2gVpbWO0sdNDMHCjgLEzXHQCRLXMpnasNa82cFy3WVzktfXsN3BKyqyTAuu72pedA7--0dH6M1lnxAA Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 5be47c3b2c4c4fc2a1db117433cbcfc4 Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-TestShelveInstance-1066410568 Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:29:44.522223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:29:44.523929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_5be47c3b2c4c4fc2a1db117433cbcfc4"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5be47c3b2c4c4fc2a1db117433cbcfc4"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/5be47c3b2c4c4fc2a1db117433cbcfc4"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/5be47c3b2c4c4fc2a1db117433cbcfc4"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:44.523929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-TestShelveInstance-1066410568 Sep 08 14:29:44.523929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 5be47c3b2c4c4fc2a1db117433cbcfc4 Sep 08 14:29:44.523929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-TestShelveInstance-1066410568 Sep 08 14:29:44.523929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-TestShelveInstance-1066410568-project Sep 08 14:29:44.523929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:29:44.523929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:29:44.523929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: f2026170f4674b63a37534c14689bd10 Sep 08 14:29:44.523929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-TestShelveInstance-1066410568-project Sep 08 14:29:44.523929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:29:44.523929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance d1062abf-7525-48c5-a9cb-91da3855fd1b could not be found. Sep 08 14:29:44.523929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'd1062abf-7525-48c5-a9cb-91da3855fd1b'} Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d1062abf-7525-48c5-a9cb-91da3855fd1b could not be found. Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:44.524931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:44.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:44.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:29:44.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:29:44.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:29:44.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:29:44.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:29:44.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:29:44.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d1062abf-7525-48c5-a9cb-91da3855fd1b could not be found. Sep 08 14:29:44.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:44.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:44.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:44.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:44.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:44.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:44.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:44.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:44.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d1062abf-7525-48c5-a9cb-91da3855fd1b could not be found. Sep 08 14:29:44.526576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:29:44.527946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c5dbed6c-c01c-4ca8-8817-d28af7089861 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Returning 404 to user: Instance d1062abf-7525-48c5-a9cb-91da3855fd1b could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:44.527946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c5dbed6c-c01c-4ca8-8817-d28af7089861 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b" status: 404 len: 111 microversion: 2.1 time: 0.035899 Sep 08 14:29:44.527946 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1404/2527] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:44 2022] GET /compute/v2.1/servers/d1062abf-7525-48c5-a9cb-91da3855fd1b => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:29:44.915445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-213fe901-cd16-4cd3-a2e9-0e757da17359 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Action: 'create', calling method: .version_select at 0x7fce0232d5e0>, body: {"keypair": {"name": "tempest-keypair-884379298", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBEJBBx4Tc9V9lK4w9J29q4MRpoqtc/aG1GZ22NTn/x7ILMw3jS8Q0Iar0kRZdL28Y1Zh37HZjQ+Xlda/621kvmhWJvVjwSVK6HWrE3h/0U1hUFJ5V4nrqFd2DU5Nxe1nqw=="}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:29:44.934380 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-213fe901-cd16-4cd3-a2e9-0e757da17359 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Getting quotas for project 4c194dd986d442d3bcaa83442a49ff70. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 14:29:44.938475 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-213fe901-cd16-4cd3-a2e9-0e757da17359 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Getting quotas for user e9925313d1d446fa81a00cb55348dff7 and project 4c194dd986d442d3bcaa83442a49ff70. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 14:29:45.060088 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-213fe901-cd16-4cd3-a2e9-0e757da17359 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 201 len: 400 microversion: 2.20 time: 0.148582 Sep 08 14:29:45.068592 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1124/2528] 10.222.0.25 () {62 vars in 1200 bytes} [Thu Sep 8 14:29:44 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 153 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 08 14:29:45.216750 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2c257fe4-7ffb-45cd-8a37-a1e69b41a66c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:45.219048 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2c257fe4-7ffb-45cd-8a37-a1e69b41a66c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:45.228684 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c257fe4-7ffb-45cd-8a37-a1e69b41a66c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:45.229677 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c257fe4-7ffb-45cd-8a37-a1e69b41a66c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:45.232283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c257fe4-7ffb-45cd-8a37-a1e69b41a66c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:45.243788 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c257fe4-7ffb-45cd-8a37-a1e69b41a66c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:45.243788 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c257fe4-7ffb-45cd-8a37-a1e69b41a66c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:45.243788 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2c257fe4-7ffb-45cd-8a37-a1e69b41a66c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:45.249612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-33131d9a-bb92-4816-b536-98242010c5c6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] Calling method '.version_select at 0x7fce021aeaf0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:45.362248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-33131d9a-bb92-4816-b536-98242010c5c6 tempest-TestShelveInstance-1066410568 tempest-TestShelveInstance-1066410568-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1782896132" status: 202 len: 0 microversion: 2.1 time: 0.116475 Sep 08 14:29:45.371385 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1125/2529] 10.222.0.25 () {58 vars in 1254 bytes} [Thu Sep 8 14:29:45 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1782896132 => generated 0 bytes in 127 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:29:45.423376 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6de38578-0f41-4159-8327-09f5381e1ea0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:45.423999 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6de38578-0f41-4159-8327-09f5381e1ea0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:45.442604 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2c257fe4-7ffb-45cd-8a37-a1e69b41a66c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:45.451736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2c257fe4-7ffb-45cd-8a37-a1e69b41a66c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1471 microversion: 2.1 time: 0.240757 Sep 08 14:29:45.452493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1405/2530] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:45 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1471 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:29:45.453060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6de38578-0f41-4159-8327-09f5381e1ea0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:45.453620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6de38578-0f41-4159-8327-09f5381e1ea0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:45.454068 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6de38578-0f41-4159-8327-09f5381e1ea0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:45.454872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6de38578-0f41-4159-8327-09f5381e1ea0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:45.457434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6de38578-0f41-4159-8327-09f5381e1ea0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:45.460098 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6de38578-0f41-4159-8327-09f5381e1ea0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:45.607857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-21515ac9-7564-4a21-877d-803e9f04e707 req-0af145e6-c2e3-481e-bace-6800b80990cb service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7ec70561-b436-4d03-bd60-6c5ae232b492", "name": "network-vif-deleted", "tag": "238b9d36-b260-4554-b58e-6b5beab1736c"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:29:45.618424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-21515ac9-7564-4a21-877d-803e9f04e707 req-0af145e6-c2e3-481e-bace-6800b80990cb service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:45.619082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-21515ac9-7564-4a21-877d-803e9f04e707 req-0af145e6-c2e3-481e-bace-6800b80990cb service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:45.619769 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-21515ac9-7564-4a21-877d-803e9f04e707 req-0af145e6-c2e3-481e-bace-6800b80990cb service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:45.633305 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-21515ac9-7564-4a21-877d-803e9f04e707 req-0af145e6-c2e3-481e-bace-6800b80990cb service nova] Creating event network-vif-deleted:238b9d36-b260-4554-b58e-6b5beab1736c for instance 7ec70561-b436-4d03-bd60-6c5ae232b492 on hv-cinder-85646 Sep 08 14:29:45.644060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-21515ac9-7564-4a21-877d-803e9f04e707 req-0af145e6-c2e3-481e-bace-6800b80990cb service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040039 Sep 08 14:29:45.644738 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1406/2531] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:29:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:29:45.693028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6de38578-0f41-4159-8327-09f5381e1ea0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:45.703025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6de38578-0f41-4159-8327-09f5381e1ea0 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1552 microversion: 2.1 time: 0.285767 Sep 08 14:29:45.703612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1126/2532] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:45 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1552 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:29:46.006471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-1a27d76f-eacc-4fde-837f-07b672a0fe8d req-29850760-af70-4b26-9c18-7bae7fd3622b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "d7ae43e4-11c8-417c-be73-4f9fb5c42f2c", "name": "network-vif-deleted", "tag": "48757fc0-370f-4fb9-a0d2-0688efed54e8"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:29:46.016781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-1a27d76f-eacc-4fde-837f-07b672a0fe8d req-29850760-af70-4b26-9c18-7bae7fd3622b service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:46.017257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-1a27d76f-eacc-4fde-837f-07b672a0fe8d req-29850760-af70-4b26-9c18-7bae7fd3622b service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:46.017907 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-1a27d76f-eacc-4fde-837f-07b672a0fe8d req-29850760-af70-4b26-9c18-7bae7fd3622b service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:46.029770 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-1a27d76f-eacc-4fde-837f-07b672a0fe8d req-29850760-af70-4b26-9c18-7bae7fd3622b service nova] Creating event network-vif-deleted:48757fc0-370f-4fb9-a0d2-0688efed54e8 for instance d7ae43e4-11c8-417c-be73-4f9fb5c42f2c on hv-cinder-85646 Sep 08 14:29:46.035129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-1a27d76f-eacc-4fde-837f-07b672a0fe8d req-29850760-af70-4b26-9c18-7bae7fd3622b service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.094206 Sep 08 14:29:46.035893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1407/2533] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:29:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:29:46.486873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e4e8afe5-4637-48a5-ad64-95505b53c215 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:46.487973 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e4e8afe5-4637-48a5-ad64-95505b53c215 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:46.503715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4e8afe5-4637-48a5-ad64-95505b53c215 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:46.504294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4e8afe5-4637-48a5-ad64-95505b53c215 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:46.504799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4e8afe5-4637-48a5-ad64-95505b53c215 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:46.505839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4e8afe5-4637-48a5-ad64-95505b53c215 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:46.506453 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4e8afe5-4637-48a5-ad64-95505b53c215 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:46.506983 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4e8afe5-4637-48a5-ad64-95505b53c215 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:46.558082 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4e8afe5-4637-48a5-ad64-95505b53c215 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:46.558615 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4e8afe5-4637-48a5-ad64-95505b53c215 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:46.559221 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e4e8afe5-4637-48a5-ad64-95505b53c215 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:46.560358 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-e4e8afe5-4637-48a5-ad64-95505b53c215 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lazy-loading 'fault' on Instance uuid 7ec70561-b436-4d03-bd60-6c5ae232b492 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:29:46.624885 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e4e8afe5-4637-48a5-ad64-95505b53c215 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:46.628859 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=4,nova_cell1:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:29:46.635154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e4e8afe5-4637-48a5-ad64-95505b53c215 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 200 len: 1309 microversion: 2.1 time: 0.154079 Sep 08 14:29:46.638440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1127/2534] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:46 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 1309 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:29:46.727789 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-eae73a13-0344-483a-af46-98bdac096376 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:46.728574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-eae73a13-0344-483a-af46-98bdac096376 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:46.735430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eae73a13-0344-483a-af46-98bdac096376 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:46.736017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eae73a13-0344-483a-af46-98bdac096376 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:46.736529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eae73a13-0344-483a-af46-98bdac096376 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:46.737324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eae73a13-0344-483a-af46-98bdac096376 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:46.737861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eae73a13-0344-483a-af46-98bdac096376 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:46.738402 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eae73a13-0344-483a-af46-98bdac096376 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:46.772958 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eae73a13-0344-483a-af46-98bdac096376 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:46.773562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eae73a13-0344-483a-af46-98bdac096376 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:46.774009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-eae73a13-0344-483a-af46-98bdac096376 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:46.775039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-eae73a13-0344-483a-af46-98bdac096376 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lazy-loading 'fault' on Instance uuid d7ae43e4-11c8-417c-be73-4f9fb5c42f2c {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:29:46.853734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-eae73a13-0344-483a-af46-98bdac096376 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:46.891146 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-eae73a13-0344-483a-af46-98bdac096376 tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1348 microversion: 2.1 time: 0.167870 Sep 08 14:29:46.892793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1408/2535] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:46 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1348 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:29:47.661711 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8d473d2a-399e-4969-8162-beb59fb39367 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:47.661711 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8d473d2a-399e-4969-8162-beb59fb39367 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 7ec70561-b436-4d03-bd60-6c5ae232b492] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:47.665356 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=95,nova_api:DELETE=3,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:29:47.673567 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d473d2a-399e-4969-8162-beb59fb39367 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:47.678279 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d473d2a-399e-4969-8162-beb59fb39367 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:47.678843 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d473d2a-399e-4969-8162-beb59fb39367 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:47.679594 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d473d2a-399e-4969-8162-beb59fb39367 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:47.680060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d473d2a-399e-4969-8162-beb59fb39367 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:47.680489 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8d473d2a-399e-4969-8162-beb59fb39367 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:47.716210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-8d473d2a-399e-4969-8162-beb59fb39367 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] HTTP exception thrown: Instance 7ec70561-b436-4d03-bd60-6c5ae232b492 could not be found. Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8d473d2a-399e-4969-8162-beb59fb39367 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Request method failure captured: Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 HTTP/1.1 Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfSNyJo8zI7plzmoT3joT1LB7boaKVSoM2Btn54mkWpdqceI6SLQFx4_Rpb5l-mqzqX661NrDxpQinv-CjzSmbNBVW650b02MoLz_tOe3RLJyylDgcU4AMsS5LTkC_9jOXC-JdkvqUTGjOl36FG-vYox8h_LoJX8tIcUOcoYc5HdH0s Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:29:47.717781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:29:47.725759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:47.725759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:29:47.725759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:29:47.725759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:29:47.725759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:29:47.725759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:29:47.725759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:29:47.725759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 5ae9377cf36e4086aa38c07b021fdead Sep 08 14:29:47.725759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:29:47.725759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:29:47.725759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 7ec70561-b436-4d03-bd60-6c5ae232b492 could not be found. Sep 08 14:29:47.725759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '7ec70561-b436-4d03-bd60-6c5ae232b492'} Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7ec70561-b436-4d03-bd60-6c5ae232b492 could not be found. Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:47.726816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:47.728620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:47.728620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:29:47.728620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:29:47.728620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:29:47.728620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:29:47.728620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:29:47.728620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:29:47.728620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7ec70561-b436-4d03-bd60-6c5ae232b492 could not be found. Sep 08 14:29:47.728620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:47.728620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:47.728620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:47.728620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:47.728620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:47.728620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:47.728620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:47.728620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:47.728620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7ec70561-b436-4d03-bd60-6c5ae232b492 could not be found. Sep 08 14:29:47.728620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:47.729962 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8d473d2a-399e-4969-8162-beb59fb39367 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Returning 404 to user: Instance 7ec70561-b436-4d03-bd60-6c5ae232b492 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:47.729962 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8d473d2a-399e-4969-8162-beb59fb39367 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492" status: 404 len: 111 microversion: 2.1 time: 0.062420 Sep 08 14:29:47.729962 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1128/2536] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:47 2022] GET /compute/v2.1/servers/7ec70561-b436-4d03-bd60-6c5ae232b492 => generated 111 bytes in 69 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:29:47.920101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f6fd05b9-9611-4f14-9f20-ab10a74544ac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:47.920816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f6fd05b9-9611-4f14-9f20-ab10a74544ac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:47.938550 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f6fd05b9-9611-4f14-9f20-ab10a74544ac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:47.939353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f6fd05b9-9611-4f14-9f20-ab10a74544ac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:47.939969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f6fd05b9-9611-4f14-9f20-ab10a74544ac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:47.940897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f6fd05b9-9611-4f14-9f20-ab10a74544ac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:47.941544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f6fd05b9-9611-4f14-9f20-ab10a74544ac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:47.942177 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f6fd05b9-9611-4f14-9f20-ab10a74544ac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:47.991173 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f6fd05b9-9611-4f14-9f20-ab10a74544ac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:47.991855 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f6fd05b9-9611-4f14-9f20-ab10a74544ac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:47.992594 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f6fd05b9-9611-4f14-9f20-ab10a74544ac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:47.993244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-f6fd05b9-9611-4f14-9f20-ab10a74544ac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lazy-loading 'fault' on Instance uuid d7ae43e4-11c8-417c-be73-4f9fb5c42f2c {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:29:48.293119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f6fd05b9-9611-4f14-9f20-ab10a74544ac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1f10a95e49744a6dbb73e54e00d81add', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '179292c42f8e4041adc935c80dba0f70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:48.310736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f6fd05b9-9611-4f14-9f20-ab10a74544ac tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 200 len: 1348 microversion: 2.1 time: 0.397515 Sep 08 14:29:48.311308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1409/2537] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:47 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 1348 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:29:49.338799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2b8c2ed1-e06b-463c-a6f7-702387737a2a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:49.341303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2b8c2ed1-e06b-463c-a6f7-702387737a2a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] [instance: d7ae43e4-11c8-417c-be73-4f9fb5c42f2c] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:49.352786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b8c2ed1-e06b-463c-a6f7-702387737a2a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:49.353617 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b8c2ed1-e06b-463c-a6f7-702387737a2a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:49.355250 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b8c2ed1-e06b-463c-a6f7-702387737a2a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:49.356114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b8c2ed1-e06b-463c-a6f7-702387737a2a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:49.356782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b8c2ed1-e06b-463c-a6f7-702387737a2a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:49.357211 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b8c2ed1-e06b-463c-a6f7-702387737a2a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:49.375920 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-2b8c2ed1-e06b-463c-a6f7-702387737a2a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] HTTP exception thrown: Instance d7ae43e4-11c8-417c-be73-4f9fb5c42f2c could not be found. Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2b8c2ed1-e06b-463c-a6f7-702387737a2a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Request method failure captured: Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c HTTP/1.1 Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfSsEI3OQCjaT8uW9n_C9Vd1UPOl7hGg4pp7-4nDKa37scg2VH8UGybcHZqyDn5ybTSjRS0vhV5EoGnyfNs_A0uLCWs9ZtiXuHPkrFMPvuwPxHSIVoo5s0Kpv1P1uWF7oFwsg1AH73mWHbrcQqseds7NX0AeiotGfcerp1SS6-JaE8A Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:29:49.383268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:29:49.385087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_179292c42f8e4041adc935c80dba0f70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/179292c42f8e4041adc935c80dba0f70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:29:49.385087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:49.385087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 179292c42f8e4041adc935c80dba0f70 Sep 08 14:29:49.385087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-TestMinimumBasicScenario-748913513 Sep 08 14:29:49.385087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:29:49.385087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:29:49.385087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:29:49.385087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 1f10a95e49744a6dbb73e54e00d81add Sep 08 14:29:49.385087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-TestMinimumBasicScenario-748913513-project Sep 08 14:29:49.385087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:29:49.385087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance d7ae43e4-11c8-417c-be73-4f9fb5c42f2c could not be found. Sep 08 14:29:49.385087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'd7ae43e4-11c8-417c-be73-4f9fb5c42f2c'} Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance d7ae43e4-11c8-417c-be73-4f9fb5c42f2c could not be found. Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:29:49.386064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:29:49.388693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:29:49.388693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:29:49.388693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:29:49.388693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:29:49.388693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:29:49.388693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:29:49.388693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:29:49.388693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance d7ae43e4-11c8-417c-be73-4f9fb5c42f2c could not be found. Sep 08 14:29:49.388693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:49.388693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:29:49.388693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:49.388693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:29:49.388693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:29:49.388693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:29:49.388693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:29:49.388693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:29:49.388693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance d7ae43e4-11c8-417c-be73-4f9fb5c42f2c could not be found. Sep 08 14:29:49.388693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:29:49.390937 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2b8c2ed1-e06b-463c-a6f7-702387737a2a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Returning 404 to user: Instance d7ae43e4-11c8-417c-be73-4f9fb5c42f2c could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:29:49.390937 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2b8c2ed1-e06b-463c-a6f7-702387737a2a tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c" status: 404 len: 111 microversion: 2.1 time: 0.046925 Sep 08 14:29:49.390937 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1129/2538] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:49 2022] GET /compute/v2.1/servers/d7ae43e4-11c8-417c-be73-4f9fb5c42f2c => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:29:49.396395 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e42162ef-2432-4fdc-a290-f26ffc7b584d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] Calling method '.version_select at 0x7fce0587b4c0>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:49.521961 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e42162ef-2432-4fdc-a290-f26ffc7b584d tempest-TestMinimumBasicScenario-748913513 tempest-TestMinimumBasicScenario-748913513-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-279563135" status: 202 len: 0 microversion: 2.1 time: 0.131104 Sep 08 14:29:49.522621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1410/2539] 10.222.0.25 () {58 vars in 1269 bytes} [Thu Sep 8 14:29:49 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-279563135 => generated 0 bytes in 139 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:29:49.615525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a1ee7c15-d272-47ae-9929-b5779825d521 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-795787648", "imageRef": "20b30425-92e5-4d78-8502-45557526ea49", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "a265cc60-700e-4951-9295-70d02eb6ee91"}], "security_groups": [{"name": "tempest-securitygroup--727690006"}], "key_name": "tempest-keypair-1145077434", "user_data": "}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:29:49.961168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.network.neutron [None req-a1ee7c15-d272-47ae-9929-b5779825d521 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] validate_networks() for [('a265cc60-700e-4951-9295-70d02eb6ee91', None, None, None, None, None)] {{(pid=127602) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 14:29:50.220447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4bdef5af-74e4-4437-8848-c80ea743b01a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1632119965", "imageRef": "20b30425-92e5-4d78-8502-45557526ea49", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "5ee979e1-44f7-4b47-8a29-595077d970e8"}], "security_groups": [{"name": "tempest-securitygroup--21940020"}], "key_name": "tempest-keypair-884379298", "user_data": "}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:29:50.468583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.network.neutron [None req-4bdef5af-74e4-4437-8848-c80ea743b01a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] validate_networks() for [('5ee979e1-44f7-4b47-8a29-595077d970e8', None, None, None, None, None)] {{(pid=127601) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 14:29:50.566042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-a1ee7c15-d272-47ae-9929-b5779825d521 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Flavor limits 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 14:29:50.566692 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-a1ee7c15-d272-47ae-9929-b5779825d521 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Image limits 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 14:29:50.567182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-a1ee7c15-d272-47ae-9929-b5779825d521 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Flavor pref 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 14:29:50.567724 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-a1ee7c15-d272-47ae-9929-b5779825d521 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Image pref 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 14:29:50.568133 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-a1ee7c15-d272-47ae-9929-b5779825d521 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 14:29:50.580126 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1ee7c15-d272-47ae-9929-b5779825d521 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:50.581289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1ee7c15-d272-47ae-9929-b5779825d521 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:50.581556 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1ee7c15-d272-47ae-9929-b5779825d521 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:50.582522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1ee7c15-d272-47ae-9929-b5779825d521 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:50.586604 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1ee7c15-d272-47ae-9929-b5779825d521 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:50.586604 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1ee7c15-d272-47ae-9929-b5779825d521 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:50.609474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-a1ee7c15-d272-47ae-9929-b5779825d521 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Getting quotas for project a4e0fba6cab34c429c73fcfac7ebfd94. Resources: {'cores', 'instances', 'ram'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 14:29:50.615532 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-a1ee7c15-d272-47ae-9929-b5779825d521 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Getting quotas for user eca89f435299415c8448931e73f5b0fc and project a4e0fba6cab34c429c73fcfac7ebfd94. Resources: {'cores', 'instances', 'ram'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 14:29:50.635741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a1ee7c15-d272-47ae-9929-b5779825d521 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Going to run 1 instances... {{(pid=127602) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 14:29:50.648567 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a1ee7c15-d272-47ae-9929-b5779825d521 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='20b30425-92e5-4d78-8502-45557526ea49',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127602) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 14:29:50.828132 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a1ee7c15-d272-47ae-9929-b5779825d521 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.224171 Sep 08 14:29:50.829803 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1130/2540] 10.222.0.25 () {60 vars in 1147 bytes} [Thu Sep 8 14:29:49 2022] POST /compute/v2.1/servers => generated 401 bytes in 1227 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 08 14:29:50.842715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-950311f4-dfb8-45ac-ba11-17e32d41108f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:50.843998 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-950311f4-dfb8-45ac-ba11-17e32d41108f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:50.974713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-950311f4-dfb8-45ac-ba11-17e32d41108f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:50.983840 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-950311f4-dfb8-45ac-ba11-17e32d41108f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1241 microversion: 2.1 time: 0.145319 Sep 08 14:29:51.000416 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1131/2541] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:50 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1241 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:29:51.819974 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-4bdef5af-74e4-4437-8848-c80ea743b01a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Flavor limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 14:29:51.820609 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-4bdef5af-74e4-4437-8848-c80ea743b01a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Image limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 14:29:51.821187 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-4bdef5af-74e4-4437-8848-c80ea743b01a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Flavor pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 14:29:51.821670 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-4bdef5af-74e4-4437-8848-c80ea743b01a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Image pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 14:29:51.822350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-4bdef5af-74e4-4437-8848-c80ea743b01a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 14:29:51.832666 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4bdef5af-74e4-4437-8848-c80ea743b01a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:51.833178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4bdef5af-74e4-4437-8848-c80ea743b01a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:51.833685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4bdef5af-74e4-4437-8848-c80ea743b01a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:51.834342 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4bdef5af-74e4-4437-8848-c80ea743b01a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:51.834799 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4bdef5af-74e4-4437-8848-c80ea743b01a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:51.835227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4bdef5af-74e4-4437-8848-c80ea743b01a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:51.859568 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-4bdef5af-74e4-4437-8848-c80ea743b01a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Getting quotas for project 4c194dd986d442d3bcaa83442a49ff70. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 14:29:51.863029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-4bdef5af-74e4-4437-8848-c80ea743b01a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Getting quotas for user e9925313d1d446fa81a00cb55348dff7 and project 4c194dd986d442d3bcaa83442a49ff70. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 14:29:51.875056 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4bdef5af-74e4-4437-8848-c80ea743b01a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Going to run 1 instances... {{(pid=127601) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 14:29:51.877377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4bdef5af-74e4-4437-8848-c80ea743b01a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='20b30425-92e5-4d78-8502-45557526ea49',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127601) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 14:29:51.912064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=95,nova_api:DELETE=2,nova_api:UPDATE=1,nova_api:INSERT=2 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:29:52.010041 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-438961b4-b3d1-4807-bcde-05e1533e8c73 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:52.011659 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-438961b4-b3d1-4807-bcde-05e1533e8c73 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:52.048589 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4bdef5af-74e4-4437-8848-c80ea743b01a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.20 time: 1.833493 Sep 08 14:29:52.050417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1411/2542] 10.222.0.25 () {62 vars in 1188 bytes} [Thu Sep 8 14:29:50 2022] POST /compute/v2.1/servers => generated 400 bytes in 1836 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) Sep 08 14:29:52.063064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9f0d32e9-aacc-4005-be8f-f0e385aebe12 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:52.064011 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9f0d32e9-aacc-4005-be8f-f0e385aebe12 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:52.094127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-438961b4-b3d1-4807-bcde-05e1533e8c73 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:52.118923 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-438961b4-b3d1-4807-bcde-05e1533e8c73 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1241 microversion: 2.1 time: 0.112623 Sep 08 14:29:52.122289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1132/2543] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:52 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1241 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:29:52.166973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9f0d32e9-aacc-4005-be8f-f0e385aebe12 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:52.180072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9f0d32e9-aacc-4005-be8f-f0e385aebe12 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:52.181787 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9f0d32e9-aacc-4005-be8f-f0e385aebe12 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:52.187628 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9f0d32e9-aacc-4005-be8f-f0e385aebe12 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1281 microversion: 2.20 time: 0.128615 Sep 08 14:29:52.191699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1412/2544] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:29:52 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1281 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:29:53.146705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3950ba42-a189-496c-9321-121566ed36cb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:53.148131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3950ba42-a189-496c-9321-121566ed36cb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:53.163493 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3950ba42-a189-496c-9321-121566ed36cb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:53.165860 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3950ba42-a189-496c-9321-121566ed36cb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:53.180743 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3950ba42-a189-496c-9321-121566ed36cb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:53.182471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3950ba42-a189-496c-9321-121566ed36cb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:53.188006 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3950ba42-a189-496c-9321-121566ed36cb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:53.191628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3950ba42-a189-496c-9321-121566ed36cb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:53.220452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1c9def69-fd83-4eff-bdba-c93543f81c2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:53.221578 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1c9def69-fd83-4eff-bdba-c93543f81c2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:53.321417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1c9def69-fd83-4eff-bdba-c93543f81c2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:53.336176 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1c9def69-fd83-4eff-bdba-c93543f81c2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:53.338561 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1c9def69-fd83-4eff-bdba-c93543f81c2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:53.343349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1c9def69-fd83-4eff-bdba-c93543f81c2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1281 microversion: 2.20 time: 0.127760 Sep 08 14:29:53.366058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3950ba42-a189-496c-9321-121566ed36cb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:53.367121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1413/2545] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:29:53 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1281 bytes in 144 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:29:53.368498 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3950ba42-a189-496c-9321-121566ed36cb tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1293 microversion: 2.1 time: 0.227165 Sep 08 14:29:53.370407 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1133/2546] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:53 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1293 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:29:53.372351 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e82a1c1b-02c8-4a42-a7e5-a285621c57ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:53.373049 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e82a1c1b-02c8-4a42-a7e5-a285621c57ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:53.380629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e82a1c1b-02c8-4a42-a7e5-a285621c57ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:53.381410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e82a1c1b-02c8-4a42-a7e5-a285621c57ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:53.382264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e82a1c1b-02c8-4a42-a7e5-a285621c57ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:53.383261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e82a1c1b-02c8-4a42-a7e5-a285621c57ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:53.384000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e82a1c1b-02c8-4a42-a7e5-a285621c57ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:53.384621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e82a1c1b-02c8-4a42-a7e5-a285621c57ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:53.417801 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e82a1c1b-02c8-4a42-a7e5-a285621c57ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:29:53.749883 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e82a1c1b-02c8-4a42-a7e5-a285621c57ca tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "DELETE /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 204 len: 0 microversion: 2.1 time: 0.389247 Sep 08 14:29:53.750574 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1414/2547] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:29:53 2022] DELETE /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 0 bytes in 391 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:29:53.760724 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e6003372-b406-40e3-a6e1-6a1e60b8a21a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:53.762530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e6003372-b406-40e3-a6e1-6a1e60b8a21a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:53.770872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6003372-b406-40e3-a6e1-6a1e60b8a21a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:53.771420 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6003372-b406-40e3-a6e1-6a1e60b8a21a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:53.771860 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6003372-b406-40e3-a6e1-6a1e60b8a21a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:53.772568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6003372-b406-40e3-a6e1-6a1e60b8a21a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:53.773510 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6003372-b406-40e3-a6e1-6a1e60b8a21a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:53.774701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6003372-b406-40e3-a6e1-6a1e60b8a21a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:54.094620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e6003372-b406-40e3-a6e1-6a1e60b8a21a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:54.104797 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e6003372-b406-40e3-a6e1-6a1e60b8a21a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1506 microversion: 2.1 time: 0.349821 Sep 08 14:29:54.106333 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1134/2548] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:53 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1506 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:29:54.374881 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-669d02b4-02a9-48b1-aca8-59666d445ab0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:54.376658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-669d02b4-02a9-48b1-aca8-59666d445ab0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:54.385239 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-669d02b4-02a9-48b1-aca8-59666d445ab0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:54.386025 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-669d02b4-02a9-48b1-aca8-59666d445ab0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:54.386966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-669d02b4-02a9-48b1-aca8-59666d445ab0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:54.394102 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-30a3041a-aabf-4747-866b-b2cfb61e7087 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:54.395514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-30a3041a-aabf-4747-866b-b2cfb61e7087 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:54.401014 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-669d02b4-02a9-48b1-aca8-59666d445ab0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:54.401949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-669d02b4-02a9-48b1-aca8-59666d445ab0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:54.402666 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-669d02b4-02a9-48b1-aca8-59666d445ab0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:54.408780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-30a3041a-aabf-4747-866b-b2cfb61e7087 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:54.409549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-30a3041a-aabf-4747-866b-b2cfb61e7087 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:54.410170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-30a3041a-aabf-4747-866b-b2cfb61e7087 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:54.414987 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-30a3041a-aabf-4747-866b-b2cfb61e7087 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:54.415751 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-30a3041a-aabf-4747-866b-b2cfb61e7087 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:54.418879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-30a3041a-aabf-4747-866b-b2cfb61e7087 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:54.629710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-30a3041a-aabf-4747-866b-b2cfb61e7087 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:54.640099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-30a3041a-aabf-4747-866b-b2cfb61e7087 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1311 microversion: 2.1 time: 0.249986 Sep 08 14:29:54.645210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1135/2549] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:54 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1311 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:29:54.702342 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-669d02b4-02a9-48b1-aca8-59666d445ab0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:54.714080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-669d02b4-02a9-48b1-aca8-59666d445ab0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:54.716142 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-669d02b4-02a9-48b1-aca8-59666d445ab0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:54.717804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-669d02b4-02a9-48b1-aca8-59666d445ab0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1333 microversion: 2.20 time: 0.348107 Sep 08 14:29:54.719832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1415/2550] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:29:54 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1333 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:29:55.129487 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-700c8d4e-651a-4be7-ade4-07df24cd7825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:55.130241 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-700c8d4e-651a-4be7-ade4-07df24cd7825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:55.137453 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-700c8d4e-651a-4be7-ade4-07df24cd7825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:55.138456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-700c8d4e-651a-4be7-ade4-07df24cd7825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:55.139078 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-700c8d4e-651a-4be7-ade4-07df24cd7825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:55.139865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-700c8d4e-651a-4be7-ade4-07df24cd7825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:55.140492 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-700c8d4e-651a-4be7-ade4-07df24cd7825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:55.140950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-700c8d4e-651a-4be7-ade4-07df24cd7825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:55.489138 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-700c8d4e-651a-4be7-ade4-07df24cd7825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:55.500736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-700c8d4e-651a-4be7-ade4-07df24cd7825 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1506 microversion: 2.1 time: 0.375929 Sep 08 14:29:55.501358 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1136/2551] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:55 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1506 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:29:55.664864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-440625d3-08d2-4168-8413-5400b925d917 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:55.666177 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-440625d3-08d2-4168-8413-5400b925d917 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:55.676156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-440625d3-08d2-4168-8413-5400b925d917 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:55.678763 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-440625d3-08d2-4168-8413-5400b925d917 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:55.679435 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-440625d3-08d2-4168-8413-5400b925d917 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:55.680334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-440625d3-08d2-4168-8413-5400b925d917 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:55.687748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-440625d3-08d2-4168-8413-5400b925d917 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:55.688445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-440625d3-08d2-4168-8413-5400b925d917 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:55.694647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:UPDATE=7,nova_cell1:SELECT=92,nova_cell1:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:29:55.749887 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ebf78a33-ff63-4a03-9fa7-0d7b2c6482c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:55.750511 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ebf78a33-ff63-4a03-9fa7-0d7b2c6482c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:55.773055 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ebf78a33-ff63-4a03-9fa7-0d7b2c6482c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:55.773816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ebf78a33-ff63-4a03-9fa7-0d7b2c6482c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:55.774479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ebf78a33-ff63-4a03-9fa7-0d7b2c6482c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:55.775349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ebf78a33-ff63-4a03-9fa7-0d7b2c6482c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:55.775960 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ebf78a33-ff63-4a03-9fa7-0d7b2c6482c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:55.776547 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ebf78a33-ff63-4a03-9fa7-0d7b2c6482c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:55.845782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-440625d3-08d2-4168-8413-5400b925d917 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:55.858068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-440625d3-08d2-4168-8413-5400b925d917 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1299 microversion: 2.1 time: 0.197566 Sep 08 14:29:55.859278 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1416/2552] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:55 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1299 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:29:55.956646 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ebf78a33-ff63-4a03-9fa7-0d7b2c6482c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:55.976423 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ebf78a33-ff63-4a03-9fa7-0d7b2c6482c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:55.986688 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ebf78a33-ff63-4a03-9fa7-0d7b2c6482c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:55.992091 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ebf78a33-ff63-4a03-9fa7-0d7b2c6482c6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1341 microversion: 2.20 time: 0.242674 Sep 08 14:29:55.992323 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1137/2553] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:29:55 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1341 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:29:56.258073 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-11ad9bd0-0970-477a-b6c2-bc5d8bf3c4b0 req-4e82f2b0-94da-49f2-a842-fbe10e3fabcc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a", "name": "network-vif-deleted", "tag": "a6d2c241-c0b3-4218-8498-e49114f1540f"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:29:56.288189 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-11ad9bd0-0970-477a-b6c2-bc5d8bf3c4b0 req-4e82f2b0-94da-49f2-a842-fbe10e3fabcc service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:56.291246 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-11ad9bd0-0970-477a-b6c2-bc5d8bf3c4b0 req-4e82f2b0-94da-49f2-a842-fbe10e3fabcc service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:56.294764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-11ad9bd0-0970-477a-b6c2-bc5d8bf3c4b0 req-4e82f2b0-94da-49f2-a842-fbe10e3fabcc service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:56.325462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-11ad9bd0-0970-477a-b6c2-bc5d8bf3c4b0 req-4e82f2b0-94da-49f2-a842-fbe10e3fabcc service nova] Creating event network-vif-deleted:a6d2c241-c0b3-4218-8498-e49114f1540f for instance dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a on hv-cinder-85646 Sep 08 14:29:56.333689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-11ad9bd0-0970-477a-b6c2-bc5d8bf3c4b0 req-4e82f2b0-94da-49f2-a842-fbe10e3fabcc service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.080221 Sep 08 14:29:56.334759 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1417/2554] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:29:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:29:56.530421 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e1d4e745-6dc8-4b7c-a34a-41353b4c545d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:56.531846 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e1d4e745-6dc8-4b7c-a34a-41353b4c545d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:56.547140 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1d4e745-6dc8-4b7c-a34a-41353b4c545d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:56.547896 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1d4e745-6dc8-4b7c-a34a-41353b4c545d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:56.548535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1d4e745-6dc8-4b7c-a34a-41353b4c545d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:56.549461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1d4e745-6dc8-4b7c-a34a-41353b4c545d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:56.552837 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1d4e745-6dc8-4b7c-a34a-41353b4c545d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:56.556082 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e1d4e745-6dc8-4b7c-a34a-41353b4c545d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:56.737850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e1d4e745-6dc8-4b7c-a34a-41353b4c545d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:56.747156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e1d4e745-6dc8-4b7c-a34a-41353b4c545d tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1464 microversion: 2.1 time: 0.223950 Sep 08 14:29:56.758752 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1138/2555] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:56 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1464 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:29:56.884546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-187c3dbb-38cc-40d0-b8f2-7af669d9ed82 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:56.885264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-187c3dbb-38cc-40d0-b8f2-7af669d9ed82 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:56.895977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-187c3dbb-38cc-40d0-b8f2-7af669d9ed82 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:56.896746 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-187c3dbb-38cc-40d0-b8f2-7af669d9ed82 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:56.897875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-187c3dbb-38cc-40d0-b8f2-7af669d9ed82 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:56.898958 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-187c3dbb-38cc-40d0-b8f2-7af669d9ed82 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:56.899669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-187c3dbb-38cc-40d0-b8f2-7af669d9ed82 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:56.900340 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-187c3dbb-38cc-40d0-b8f2-7af669d9ed82 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:57.027389 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f5cbd428-8b27-4621-bab1-ceec462237c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:57.038982 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f5cbd428-8b27-4621-bab1-ceec462237c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:57.067718 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f5cbd428-8b27-4621-bab1-ceec462237c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:57.072217 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f5cbd428-8b27-4621-bab1-ceec462237c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:57.075533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f5cbd428-8b27-4621-bab1-ceec462237c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:57.077013 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f5cbd428-8b27-4621-bab1-ceec462237c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:57.080341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f5cbd428-8b27-4621-bab1-ceec462237c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:57.081188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f5cbd428-8b27-4621-bab1-ceec462237c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:57.244897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f5cbd428-8b27-4621-bab1-ceec462237c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:57.249731 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-187c3dbb-38cc-40d0-b8f2-7af669d9ed82 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:57.255316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f5cbd428-8b27-4621-bab1-ceec462237c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:57.258021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-187c3dbb-38cc-40d0-b8f2-7af669d9ed82 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1366 microversion: 2.1 time: 0.377752 Sep 08 14:29:57.262215 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f5cbd428-8b27-4621-bab1-ceec462237c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:57.274162 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1418/2556] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:56 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1366 bytes in 384 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:29:57.282830 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f5cbd428-8b27-4621-bab1-ceec462237c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1341 microversion: 2.20 time: 0.266180 Sep 08 14:29:57.283997 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1139/2557] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:29:57 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1341 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:29:57.795174 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6f770de7-c8a4-4d32-b649-ee36d0d7ec51 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:57.802697 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6f770de7-c8a4-4d32-b649-ee36d0d7ec51 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:57.805187 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f770de7-c8a4-4d32-b649-ee36d0d7ec51 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:57.805885 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f770de7-c8a4-4d32-b649-ee36d0d7ec51 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:57.806899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f770de7-c8a4-4d32-b649-ee36d0d7ec51 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:57.808172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f770de7-c8a4-4d32-b649-ee36d0d7ec51 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:57.809671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f770de7-c8a4-4d32-b649-ee36d0d7ec51 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:57.810542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6f770de7-c8a4-4d32-b649-ee36d0d7ec51 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:57.960841 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6f770de7-c8a4-4d32-b649-ee36d0d7ec51 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:57.985882 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6f770de7-c8a4-4d32-b649-ee36d0d7ec51 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1464 microversion: 2.1 time: 0.202477 Sep 08 14:29:57.989221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1419/2558] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:57 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1464 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:29:58.288988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-31dec548-3ecd-440a-8c37-b3fed678836a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:58.290928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-31dec548-3ecd-440a-8c37-b3fed678836a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:58.313923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-98642fae-288b-4a13-b488-92b64853a4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:58.314428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31dec548-3ecd-440a-8c37-b3fed678836a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:58.314682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-98642fae-288b-4a13-b488-92b64853a4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:58.315446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31dec548-3ecd-440a-8c37-b3fed678836a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:58.316099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31dec548-3ecd-440a-8c37-b3fed678836a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:58.317033 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31dec548-3ecd-440a-8c37-b3fed678836a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:58.317751 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31dec548-3ecd-440a-8c37-b3fed678836a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:58.322344 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-31dec548-3ecd-440a-8c37-b3fed678836a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:58.336714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-98642fae-288b-4a13-b488-92b64853a4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:58.337230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-98642fae-288b-4a13-b488-92b64853a4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:58.337685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-98642fae-288b-4a13-b488-92b64853a4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:58.338438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-98642fae-288b-4a13-b488-92b64853a4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:58.338906 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-98642fae-288b-4a13-b488-92b64853a4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:58.339341 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-98642fae-288b-4a13-b488-92b64853a4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:58.565740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-98642fae-288b-4a13-b488-92b64853a4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:58.579544 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-98642fae-288b-4a13-b488-92b64853a4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:58.582084 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-98642fae-288b-4a13-b488-92b64853a4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:58.583384 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-98642fae-288b-4a13-b488-92b64853a4bf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1341 microversion: 2.20 time: 0.274583 Sep 08 14:29:58.583990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1420/2559] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:29:58 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1341 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 14:29:58.683387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-31dec548-3ecd-440a-8c37-b3fed678836a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:58.698268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-31dec548-3ecd-440a-8c37-b3fed678836a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1366 microversion: 2.1 time: 0.413681 Sep 08 14:29:58.699544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1140/2560] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:58 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1366 bytes in 416 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:29:59.013143 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8b3c63ce-7904-4866-9caa-f4535a1fb900 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:59.013841 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8b3c63ce-7904-4866-9caa-f4535a1fb900 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:59.027303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b3c63ce-7904-4866-9caa-f4535a1fb900 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:59.028820 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b3c63ce-7904-4866-9caa-f4535a1fb900 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:59.028820 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b3c63ce-7904-4866-9caa-f4535a1fb900 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:59.029380 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b3c63ce-7904-4866-9caa-f4535a1fb900 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:59.030094 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b3c63ce-7904-4866-9caa-f4535a1fb900 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:59.035629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b3c63ce-7904-4866-9caa-f4535a1fb900 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:59.137458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b3c63ce-7904-4866-9caa-f4535a1fb900 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:59.138079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b3c63ce-7904-4866-9caa-f4535a1fb900 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:59.139620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b3c63ce-7904-4866-9caa-f4535a1fb900 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:59.140258 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-8b3c63ce-7904-4866-9caa-f4535a1fb900 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lazy-loading 'fault' on Instance uuid dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:29:59.264621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-53792f67-70aa-4fa8-b0ac-5f9699c89add tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-2139721147", "imageRef": "20b30425-92e5-4d78-8502-45557526ea49", "flavorRef": "452", "networks": [{"uuid": "2fd138c6-fd4e-45ba-b3b6-39ff66a4e89f"}]}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:29:59.280904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8b3c63ce-7904-4866-9caa-f4535a1fb900 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:59.294486 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8b3c63ce-7904-4866-9caa-f4535a1fb900 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 200 len: 1311 microversion: 2.1 time: 0.285584 Sep 08 14:29:59.307472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1421/2561] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:59 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 1311 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:29:59.344892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.network.neutron [None req-53792f67-70aa-4fa8-b0ac-5f9699c89add tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] validate_networks() for [('2fd138c6-fd4e-45ba-b3b6-39ff66a4e89f', None, None, None, None, None)] {{(pid=127602) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 14:29:59.617929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ae9e05f9-89ec-4f84-9de2-55e559c8f428 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:59.619514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ae9e05f9-89ec-4f84-9de2-55e559c8f428 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:59.658199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae9e05f9-89ec-4f84-9de2-55e559c8f428 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:59.658199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae9e05f9-89ec-4f84-9de2-55e559c8f428 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:59.658199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae9e05f9-89ec-4f84-9de2-55e559c8f428 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:59.658199 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae9e05f9-89ec-4f84-9de2-55e559c8f428 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:59.666104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae9e05f9-89ec-4f84-9de2-55e559c8f428 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:59.666702 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ae9e05f9-89ec-4f84-9de2-55e559c8f428 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:59.825265 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ae9e05f9-89ec-4f84-9de2-55e559c8f428 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:59.849233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ae9e05f9-89ec-4f84-9de2-55e559c8f428 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:59.852274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ae9e05f9-89ec-4f84-9de2-55e559c8f428 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:29:59.853960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ae9e05f9-89ec-4f84-9de2-55e559c8f428 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1339 microversion: 2.20 time: 0.245779 Sep 08 14:29:59.856020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1422/2562] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:29:59 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1339 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:29:59.862767 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-805a6a68-734e-4728-9c27-605d39631d1c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:29:59.863621 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-805a6a68-734e-4728-9c27-605d39631d1c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:29:59.870678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-805a6a68-734e-4728-9c27-605d39631d1c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:59.871987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-805a6a68-734e-4728-9c27-605d39631d1c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:59.872631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-805a6a68-734e-4728-9c27-605d39631d1c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:29:59.873817 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-805a6a68-734e-4728-9c27-605d39631d1c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:29:59.874482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-805a6a68-734e-4728-9c27-605d39631d1c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:29:59.875072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-805a6a68-734e-4728-9c27-605d39631d1c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:00.291875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-805a6a68-734e-4728-9c27-605d39631d1c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:00.312692 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-805a6a68-734e-4728-9c27-605d39631d1c tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1366 microversion: 2.1 time: 0.455025 Sep 08 14:30:00.314264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1423/2563] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:29:59 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1366 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:00.338169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-df196979-1c5f-47a6-b103-c4e4630710d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:00.339060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-df196979-1c5f-47a6-b103-c4e4630710d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:00.347561 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df196979-1c5f-47a6-b103-c4e4630710d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:00.348316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df196979-1c5f-47a6-b103-c4e4630710d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:00.348955 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df196979-1c5f-47a6-b103-c4e4630710d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:00.349845 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df196979-1c5f-47a6-b103-c4e4630710d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:00.350507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df196979-1c5f-47a6-b103-c4e4630710d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:00.351201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df196979-1c5f-47a6-b103-c4e4630710d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:00.391884 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-df196979-1c5f-47a6-b103-c4e4630710d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] HTTP exception thrown: Instance dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a could not be found. Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-df196979-1c5f-47a6-b103-c4e4630710d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Request method failure captured: Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a HTTP/1.1 Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPpGZzMisIeN0rinm76oerWeI1e2n1ZCDvu1_QxGMaqly5J66d47JWYR-nVcthn5vbK6XFUxRx7ULj4gJs_JU089dQK1DqQD0zha4-7jxb_89w0QMqZgtACCW34zdnpCyIfB12dgyAWUWv38AgePo6JrTClIfdFoSLdR5OVWnj2kmY Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 1346d4105dce4c6e928fcb11cde56854 Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-188566618 Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:30:00.395371 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:30:00.397023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:30:00.397023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-ServerRescueNegativeTestJSON-188566618 Sep 08 14:30:00.397023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 1346d4105dce4c6e928fcb11cde56854 Sep 08 14:30:00.397023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-188566618 Sep 08 14:30:00.397023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-ServerRescueNegativeTestJSON-188566618-project Sep 08 14:30:00.397023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:30:00.397023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:30:00.397023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 792808dcbf894aa98c671cf43ac21b8b Sep 08 14:30:00.397023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-ServerRescueNegativeTestJSON-188566618-project Sep 08 14:30:00.397023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:30:00.397023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a could not be found. Sep 08 14:30:00.397023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a'} Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a could not be found. Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:30:00.398917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:30:00.401824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:30:00.401824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:30:00.401824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:30:00.401824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:30:00.401824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:30:00.401824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:30:00.401824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:30:00.401824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a could not be found. Sep 08 14:30:00.401824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:30:00.401824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:30:00.401824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:30:00.401824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:30:00.401824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:30:00.401824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:30:00.401824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:30:00.401824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:30:00.401824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a could not be found. Sep 08 14:30:00.401824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:30:00.405624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-df196979-1c5f-47a6-b103-c4e4630710d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Returning 404 to user: Instance dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:30:00.405624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-df196979-1c5f-47a6-b103-c4e4630710d8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a" status: 404 len: 111 microversion: 2.1 time: 0.063308 Sep 08 14:30:00.405624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1424/2564] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:00 2022] GET /compute/v2.1/servers/dd2a08a3-ad91-4a8f-a746-35a7ad2fc19a => generated 111 bytes in 67 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:30:00.417649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6a888b4c-b29f-4a8d-9f33-e2ebcb3cf6e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:00.418257 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6a888b4c-b29f-4a8d-9f33-e2ebcb3cf6e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:00.427491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a888b4c-b29f-4a8d-9f33-e2ebcb3cf6e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:00.428238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a888b4c-b29f-4a8d-9f33-e2ebcb3cf6e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:00.428887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a888b4c-b29f-4a8d-9f33-e2ebcb3cf6e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:00.429948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a888b4c-b29f-4a8d-9f33-e2ebcb3cf6e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:00.430635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a888b4c-b29f-4a8d-9f33-e2ebcb3cf6e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:00.431896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6a888b4c-b29f-4a8d-9f33-e2ebcb3cf6e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:00.492804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6a888b4c-b29f-4a8d-9f33-e2ebcb3cf6e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:30:00.599932 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=2 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:00.625904 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-53792f67-70aa-4fa8-b0ac-5f9699c89add tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Flavor limits 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 14:30:00.625904 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-53792f67-70aa-4fa8-b0ac-5f9699c89add tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Image limits 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 14:30:00.625904 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-53792f67-70aa-4fa8-b0ac-5f9699c89add tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Flavor pref 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 14:30:00.625904 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-53792f67-70aa-4fa8-b0ac-5f9699c89add tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Image pref 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 14:30:00.626447 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-53792f67-70aa-4fa8-b0ac-5f9699c89add tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 14:30:00.637168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-53792f67-70aa-4fa8-b0ac-5f9699c89add tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:00.638078 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-53792f67-70aa-4fa8-b0ac-5f9699c89add tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:00.638723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-53792f67-70aa-4fa8-b0ac-5f9699c89add tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:00.639610 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-53792f67-70aa-4fa8-b0ac-5f9699c89add tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:00.640411 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-53792f67-70aa-4fa8-b0ac-5f9699c89add tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:00.641179 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-53792f67-70aa-4fa8-b0ac-5f9699c89add tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:00.667437 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-53792f67-70aa-4fa8-b0ac-5f9699c89add tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Getting quotas for project 1363660dde8f4e4295adc44c6ab87f8b. Resources: {'cores', 'instances', 'ram'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 14:30:00.673449 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-53792f67-70aa-4fa8-b0ac-5f9699c89add tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Getting quotas for user 5ae9377cf36e4086aa38c07b021fdead and project 1363660dde8f4e4295adc44c6ab87f8b. Resources: {'cores', 'instances', 'ram'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 14:30:00.696336 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-53792f67-70aa-4fa8-b0ac-5f9699c89add tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Going to run 1 instances... {{(pid=127602) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 14:30:00.696336 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-53792f67-70aa-4fa8-b0ac-5f9699c89add tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='20b30425-92e5-4d78-8502-45557526ea49',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127602) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 14:30:00.869786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6a888b4c-b29f-4a8d-9f33-e2ebcb3cf6e6 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "DELETE /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 204 len: 0 microversion: 2.1 time: 0.460138 Sep 08 14:30:00.870744 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1425/2565] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:30:00 2022] DELETE /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 0 bytes in 463 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:30:00.877240 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-f6ce22d0-0a0a-4d9a-b6c2-42fe039872d0 req-351a40c6-72f1-47ca-9079-493b759d6c6a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bcef9a84-43d0-440b-804b-4a32893fc0d4", "tag": "c960c128-6e5a-4241-bfed-484d84bfbc84"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:30:00.900469 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-f6ce22d0-0a0a-4d9a-b6c2-42fe039872d0 req-351a40c6-72f1-47ca-9079-493b759d6c6a service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:00.901100 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-f6ce22d0-0a0a-4d9a-b6c2-42fe039872d0 req-351a40c6-72f1-47ca-9079-493b759d6c6a service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:00.901653 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-f6ce22d0-0a0a-4d9a-b6c2-42fe039872d0 req-351a40c6-72f1-47ca-9079-493b759d6c6a service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:00.903715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-53792f67-70aa-4fa8-b0ac-5f9699c89add tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.647355 Sep 08 14:30:00.910976 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1141/2566] 10.222.0.25 () {60 vars in 1147 bytes} [Thu Sep 8 14:29:59 2022] POST /compute/v2.1/servers => generated 380 bytes in 1659 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 08 14:30:00.921274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-45d362fc-befa-44bc-865d-7452115895df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:00.922123 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-45d362fc-befa-44bc-865d-7452115895df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:00.932888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-45d362fc-befa-44bc-865d-7452115895df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:00.947163 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-45d362fc-befa-44bc-865d-7452115895df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:00.947163 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-45d362fc-befa-44bc-865d-7452115895df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:00.947163 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-45d362fc-befa-44bc-865d-7452115895df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:00.947613 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-f6ce22d0-0a0a-4d9a-b6c2-42fe039872d0 req-351a40c6-72f1-47ca-9079-493b759d6c6a service nova] Creating event network-changed:c960c128-6e5a-4241-bfed-484d84bfbc84 for instance bcef9a84-43d0-440b-804b-4a32893fc0d4 on hv-cinder-85646 Sep 08 14:30:00.947721 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-45d362fc-befa-44bc-865d-7452115895df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:00.947721 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-45d362fc-befa-44bc-865d-7452115895df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:00.972757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-f6ce22d0-0a0a-4d9a-b6c2-42fe039872d0 req-351a40c6-72f1-47ca-9079-493b759d6c6a service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.100508 Sep 08 14:30:00.980570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1426/2567] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:30:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:30:00.986223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-53fdbc9f-da5e-4a84-ae80-c6f28f97d76d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:01.002987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-53fdbc9f-da5e-4a84-ae80-c6f28f97d76d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:01.021226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-53fdbc9f-da5e-4a84-ae80-c6f28f97d76d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:01.024490 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-53fdbc9f-da5e-4a84-ae80-c6f28f97d76d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:01.025124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-53fdbc9f-da5e-4a84-ae80-c6f28f97d76d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:01.026050 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-53fdbc9f-da5e-4a84-ae80-c6f28f97d76d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:01.028701 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-53fdbc9f-da5e-4a84-ae80-c6f28f97d76d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:01.030473 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-53fdbc9f-da5e-4a84-ae80-c6f28f97d76d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:01.318451 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-45d362fc-befa-44bc-865d-7452115895df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:01.327252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-45d362fc-befa-44bc-865d-7452115895df tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.414452 Sep 08 14:30:01.339594 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1142/2568] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:00 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:01.347602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-581f1566-4fe8-42b9-affe-65458d85ffe4 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:01.349076 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-581f1566-4fe8-42b9-affe-65458d85ffe4 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:01.548000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-53fdbc9f-da5e-4a84-ae80-c6f28f97d76d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:01.571322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-53fdbc9f-da5e-4a84-ae80-c6f28f97d76d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:01.571322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-53fdbc9f-da5e-4a84-ae80-c6f28f97d76d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:01.571322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-53fdbc9f-da5e-4a84-ae80-c6f28f97d76d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1405 microversion: 2.20 time: 0.587469 Sep 08 14:30:01.571728 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1427/2569] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:30:00 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1405 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:30:01.579722 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3cafdf47-ae0c-46ee-bd45-cfb185903a60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:01.580841 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3cafdf47-ae0c-46ee-bd45-cfb185903a60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:01.587655 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3cafdf47-ae0c-46ee-bd45-cfb185903a60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:01.588422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3cafdf47-ae0c-46ee-bd45-cfb185903a60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:01.589075 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3cafdf47-ae0c-46ee-bd45-cfb185903a60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:01.590817 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3cafdf47-ae0c-46ee-bd45-cfb185903a60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:01.592096 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3cafdf47-ae0c-46ee-bd45-cfb185903a60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:01.592694 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3cafdf47-ae0c-46ee-bd45-cfb185903a60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:01.805623 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-581f1566-4fe8-42b9-affe-65458d85ffe4 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:01.826394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-581f1566-4fe8-42b9-affe-65458d85ffe4 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 200 len: 1219 microversion: 2.1 time: 0.484043 Sep 08 14:30:01.826394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1143/2570] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:01 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 1219 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:01.846354 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=2 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:02.355075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a6862e8e-d028-484f-9c41-901bd33aa9b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:02.356932 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a6862e8e-d028-484f-9c41-901bd33aa9b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:02.367094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6862e8e-d028-484f-9c41-901bd33aa9b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:02.368185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6862e8e-d028-484f-9c41-901bd33aa9b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:02.368855 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6862e8e-d028-484f-9c41-901bd33aa9b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:02.371549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6862e8e-d028-484f-9c41-901bd33aa9b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:02.371549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6862e8e-d028-484f-9c41-901bd33aa9b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:02.371549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a6862e8e-d028-484f-9c41-901bd33aa9b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:02.391272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3cafdf47-ae0c-46ee-bd45-cfb185903a60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:02.403868 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3cafdf47-ae0c-46ee-bd45-cfb185903a60 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1366 microversion: 2.1 time: 0.828480 Sep 08 14:30:02.404465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1428/2571] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:01 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1366 bytes in 834 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:02.600717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4d438ec6-075b-46dd-8e5b-7f520d93ae7e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:02.601321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4d438ec6-075b-46dd-8e5b-7f520d93ae7e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:02.608661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4d438ec6-075b-46dd-8e5b-7f520d93ae7e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:02.609307 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4d438ec6-075b-46dd-8e5b-7f520d93ae7e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:02.610068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4d438ec6-075b-46dd-8e5b-7f520d93ae7e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:02.612592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4d438ec6-075b-46dd-8e5b-7f520d93ae7e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:02.612592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4d438ec6-075b-46dd-8e5b-7f520d93ae7e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:02.612592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4d438ec6-075b-46dd-8e5b-7f520d93ae7e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:02.833909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a6862e8e-d028-484f-9c41-901bd33aa9b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:02.847920 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a6862e8e-d028-484f-9c41-901bd33aa9b8 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1522 microversion: 2.1 time: 0.497358 Sep 08 14:30:02.848868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1144/2572] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:02 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1522 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:30:02.869452 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f854bc9b-e544-4fc9-861b-d178e0d65b76 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:02.869452 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f854bc9b-e544-4fc9-861b-d178e0d65b76 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:02.878265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f854bc9b-e544-4fc9-861b-d178e0d65b76 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:02.880611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f854bc9b-e544-4fc9-861b-d178e0d65b76 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:02.881026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f854bc9b-e544-4fc9-861b-d178e0d65b76 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:02.884084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f854bc9b-e544-4fc9-861b-d178e0d65b76 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:02.884734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f854bc9b-e544-4fc9-861b-d178e0d65b76 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:02.886128 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f854bc9b-e544-4fc9-861b-d178e0d65b76 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:02.897304 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4d438ec6-075b-46dd-8e5b-7f520d93ae7e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:02.907377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4d438ec6-075b-46dd-8e5b-7f520d93ae7e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:02.910130 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4d438ec6-075b-46dd-8e5b-7f520d93ae7e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:02.918075 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4d438ec6-075b-46dd-8e5b-7f520d93ae7e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1405 microversion: 2.20 time: 0.314594 Sep 08 14:30:02.921835 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1429/2573] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:30:02 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1405 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 14:30:03.052426 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f854bc9b-e544-4fc9-861b-d178e0d65b76 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:03.064278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f854bc9b-e544-4fc9-861b-d178e0d65b76 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 200 len: 1215 microversion: 2.1 time: 0.214099 Sep 08 14:30:03.065200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1145/2574] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:02 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 1215 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:03.309597 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-b29cf0dc-a937-4ab7-9adf-06ba3a2251a7 req-fc829295-0187-4735-b854-d5eae126e022 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "2707cec6-5b8c-41b5-8ae3-1b4c4de591af", "name": "network-vif-deleted", "tag": "850abdba-93ff-4f41-832b-fbb4d1e81248"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:30:03.321496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-b29cf0dc-a937-4ab7-9adf-06ba3a2251a7 req-fc829295-0187-4735-b854-d5eae126e022 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:03.323866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-b29cf0dc-a937-4ab7-9adf-06ba3a2251a7 req-fc829295-0187-4735-b854-d5eae126e022 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:03.323866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-b29cf0dc-a937-4ab7-9adf-06ba3a2251a7 req-fc829295-0187-4735-b854-d5eae126e022 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:03.337251 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=14 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:03.341700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-b29cf0dc-a937-4ab7-9adf-06ba3a2251a7 req-fc829295-0187-4735-b854-d5eae126e022 service nova] Creating event network-vif-deleted:850abdba-93ff-4f41-832b-fbb4d1e81248 for instance 2707cec6-5b8c-41b5-8ae3-1b4c4de591af on hv-cinder-85646 Sep 08 14:30:03.350026 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-b29cf0dc-a937-4ab7-9adf-06ba3a2251a7 req-fc829295-0187-4735-b854-d5eae126e022 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044650 Sep 08 14:30:03.350876 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1430/2575] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:30:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:30:03.428253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b3509369-7f02-441f-b634-c71557e9969b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:03.428942 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b3509369-7f02-441f-b634-c71557e9969b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:03.437444 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3509369-7f02-441f-b634-c71557e9969b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:03.437940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3509369-7f02-441f-b634-c71557e9969b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:03.438998 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3509369-7f02-441f-b634-c71557e9969b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:03.439189 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3509369-7f02-441f-b634-c71557e9969b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:03.439946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3509369-7f02-441f-b634-c71557e9969b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:03.440583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b3509369-7f02-441f-b634-c71557e9969b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:03.878244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0f79f2f3-aa28-443e-86a9-65ea8467b5b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:03.879541 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0f79f2f3-aa28-443e-86a9-65ea8467b5b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:03.893188 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0f79f2f3-aa28-443e-86a9-65ea8467b5b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:03.894651 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0f79f2f3-aa28-443e-86a9-65ea8467b5b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:03.897258 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0f79f2f3-aa28-443e-86a9-65ea8467b5b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:03.898864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0f79f2f3-aa28-443e-86a9-65ea8467b5b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:03.899361 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0f79f2f3-aa28-443e-86a9-65ea8467b5b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:03.899805 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0f79f2f3-aa28-443e-86a9-65ea8467b5b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:04.046472 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b3509369-7f02-441f-b634-c71557e9969b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:04.084586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b3509369-7f02-441f-b634-c71557e9969b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1366 microversion: 2.1 time: 0.660802 Sep 08 14:30:04.086452 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1146/2576] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:03 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1366 bytes in 663 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:04.091840 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-10401109-e373-476c-b498-0cb90d31e3e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:04.092816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-10401109-e373-476c-b498-0cb90d31e3e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:04.099793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-10401109-e373-476c-b498-0cb90d31e3e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:04.100383 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-10401109-e373-476c-b498-0cb90d31e3e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:04.101047 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-10401109-e373-476c-b498-0cb90d31e3e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:04.108288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-10401109-e373-476c-b498-0cb90d31e3e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:04.112686 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-10401109-e373-476c-b498-0cb90d31e3e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:04.120575 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-10401109-e373-476c-b498-0cb90d31e3e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:04.131054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0f79f2f3-aa28-443e-86a9-65ea8467b5b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:04.140691 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0f79f2f3-aa28-443e-86a9-65ea8467b5b7 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1307 microversion: 2.1 time: 0.267358 Sep 08 14:30:04.151751 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1431/2577] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:03 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1307 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:04.156625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cbe33fe2-8b37-4e0c-8a57-507a2f30e828 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:04.157237 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cbe33fe2-8b37-4e0c-8a57-507a2f30e828 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:04.173666 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbe33fe2-8b37-4e0c-8a57-507a2f30e828 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:04.174488 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbe33fe2-8b37-4e0c-8a57-507a2f30e828 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:04.175147 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbe33fe2-8b37-4e0c-8a57-507a2f30e828 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:04.176019 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbe33fe2-8b37-4e0c-8a57-507a2f30e828 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:04.186887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbe33fe2-8b37-4e0c-8a57-507a2f30e828 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:04.187608 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cbe33fe2-8b37-4e0c-8a57-507a2f30e828 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:04.300856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cbe33fe2-8b37-4e0c-8a57-507a2f30e828 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:04.311422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cbe33fe2-8b37-4e0c-8a57-507a2f30e828 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 200 len: 1271 microversion: 2.1 time: 0.158433 Sep 08 14:30:04.332870 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1432/2578] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:04 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 1271 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:04.575994 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-10401109-e373-476c-b498-0cb90d31e3e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:04.591068 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-10401109-e373-476c-b498-0cb90d31e3e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:04.597493 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-10401109-e373-476c-b498-0cb90d31e3e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:04.601147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-10401109-e373-476c-b498-0cb90d31e3e1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1405 microversion: 2.20 time: 0.512772 Sep 08 14:30:04.601992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1147/2579] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:30:04 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1405 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:30:05.111619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-992748a9-4a65-4e37-bea6-1b0fb3b81d87 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:05.113360 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-992748a9-4a65-4e37-bea6-1b0fb3b81d87 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:05.126029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-992748a9-4a65-4e37-bea6-1b0fb3b81d87 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:05.132970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-992748a9-4a65-4e37-bea6-1b0fb3b81d87 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:05.133832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-992748a9-4a65-4e37-bea6-1b0fb3b81d87 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:05.135911 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-992748a9-4a65-4e37-bea6-1b0fb3b81d87 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:05.138493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-992748a9-4a65-4e37-bea6-1b0fb3b81d87 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:05.139170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-992748a9-4a65-4e37-bea6-1b0fb3b81d87 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:05.174342 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c5179909-14d5-401c-a40c-4b12a90b7436 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:05.175744 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c5179909-14d5-401c-a40c-4b12a90b7436 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:05.189692 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:05.193006 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5179909-14d5-401c-a40c-4b12a90b7436 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:05.196645 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5179909-14d5-401c-a40c-4b12a90b7436 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:05.196645 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5179909-14d5-401c-a40c-4b12a90b7436 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:05.201425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5179909-14d5-401c-a40c-4b12a90b7436 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:05.202156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5179909-14d5-401c-a40c-4b12a90b7436 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:05.213827 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5179909-14d5-401c-a40c-4b12a90b7436 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:05.291133 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5179909-14d5-401c-a40c-4b12a90b7436 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:05.291910 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5179909-14d5-401c-a40c-4b12a90b7436 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:05.292524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c5179909-14d5-401c-a40c-4b12a90b7436 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:05.293707 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-c5179909-14d5-401c-a40c-4b12a90b7436 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lazy-loading 'fault' on Instance uuid 2707cec6-5b8c-41b5-8ae3-1b4c4de591af {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:30:05.385626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c5179909-14d5-401c-a40c-4b12a90b7436 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:05.398244 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c5179909-14d5-401c-a40c-4b12a90b7436 tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1312 microversion: 2.1 time: 0.229126 Sep 08 14:30:05.400771 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1148/2580] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:05 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1312 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:05.406507 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6c1061a2-f617-4e67-8469-26d58bc6e5a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:05.407683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6c1061a2-f617-4e67-8469-26d58bc6e5a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:05.426202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c1061a2-f617-4e67-8469-26d58bc6e5a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:05.427262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c1061a2-f617-4e67-8469-26d58bc6e5a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:05.427881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c1061a2-f617-4e67-8469-26d58bc6e5a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:05.428766 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c1061a2-f617-4e67-8469-26d58bc6e5a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:05.429430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c1061a2-f617-4e67-8469-26d58bc6e5a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:05.430333 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c1061a2-f617-4e67-8469-26d58bc6e5a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:05.491120 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-992748a9-4a65-4e37-bea6-1b0fb3b81d87 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:05.514593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6c1061a2-f617-4e67-8469-26d58bc6e5a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:05.520237 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-992748a9-4a65-4e37-bea6-1b0fb3b81d87 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1366 microversion: 2.1 time: 0.413743 Sep 08 14:30:05.523877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1433/2581] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:05 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1366 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:05.529543 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=100 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:05.533001 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6c1061a2-f617-4e67-8469-26d58bc6e5a5 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 200 len: 1289 microversion: 2.1 time: 0.130383 Sep 08 14:30:05.534616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1149/2582] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:05 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 1289 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:05.545147 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-f2348885-6a8c-47b0-b9bd-c4af7e53a9c2 req-40ffcb43-f104-4c28-bfeb-5b3305ebd056 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b0741665-d24b-4c17-93c7-90bcfff633b9", "tag": "67fe80b7-8ae7-46be-8809-bd4958623c19"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:30:05.566350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-f2348885-6a8c-47b0-b9bd-c4af7e53a9c2 req-40ffcb43-f104-4c28-bfeb-5b3305ebd056 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:05.567029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-f2348885-6a8c-47b0-b9bd-c4af7e53a9c2 req-40ffcb43-f104-4c28-bfeb-5b3305ebd056 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:05.567570 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-f2348885-6a8c-47b0-b9bd-c4af7e53a9c2 req-40ffcb43-f104-4c28-bfeb-5b3305ebd056 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:05.591237 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-f2348885-6a8c-47b0-b9bd-c4af7e53a9c2 req-40ffcb43-f104-4c28-bfeb-5b3305ebd056 service nova] Creating event network-changed:67fe80b7-8ae7-46be-8809-bd4958623c19 for instance b0741665-d24b-4c17-93c7-90bcfff633b9 on hv-cinder-85646 Sep 08 14:30:05.597094 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-f2348885-6a8c-47b0-b9bd-c4af7e53a9c2 req-40ffcb43-f104-4c28-bfeb-5b3305ebd056 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071615 Sep 08 14:30:05.599205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1434/2583] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:30:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:30:05.629174 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-53570d7c-b6a2-49f3-a553-9186ce928183 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:05.630117 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-53570d7c-b6a2-49f3-a553-9186ce928183 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:05.637017 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-53570d7c-b6a2-49f3-a553-9186ce928183 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:05.637764 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-53570d7c-b6a2-49f3-a553-9186ce928183 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:05.638392 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-53570d7c-b6a2-49f3-a553-9186ce928183 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:05.639344 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-53570d7c-b6a2-49f3-a553-9186ce928183 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:05.640076 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-53570d7c-b6a2-49f3-a553-9186ce928183 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:05.640867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-53570d7c-b6a2-49f3-a553-9186ce928183 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:05.921685 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-53570d7c-b6a2-49f3-a553-9186ce928183 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:05.941694 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-53570d7c-b6a2-49f3-a553-9186ce928183 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:05.948419 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-53570d7c-b6a2-49f3-a553-9186ce928183 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:05.966227 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-53570d7c-b6a2-49f3-a553-9186ce928183 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1405 microversion: 2.20 time: 0.328173 Sep 08 14:30:05.966227 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1150/2584] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:30:05 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1405 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:30:06.432619 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d4ecf857-385d-4556-a883-5595cf23a28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:06.434607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d4ecf857-385d-4556-a883-5595cf23a28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:06.445958 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4ecf857-385d-4556-a883-5595cf23a28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:06.448153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4ecf857-385d-4556-a883-5595cf23a28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:06.449087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4ecf857-385d-4556-a883-5595cf23a28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:06.450794 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4ecf857-385d-4556-a883-5595cf23a28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:06.451497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4ecf857-385d-4556-a883-5595cf23a28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:06.452144 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4ecf857-385d-4556-a883-5595cf23a28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:06.522576 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4ecf857-385d-4556-a883-5595cf23a28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:06.523515 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4ecf857-385d-4556-a883-5595cf23a28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:06.524238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d4ecf857-385d-4556-a883-5595cf23a28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:06.524925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-d4ecf857-385d-4556-a883-5595cf23a28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lazy-loading 'fault' on Instance uuid 2707cec6-5b8c-41b5-8ae3-1b4c4de591af {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:30:06.587174 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d4ecf857-385d-4556-a883-5595cf23a28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '792808dcbf894aa98c671cf43ac21b8b', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1346d4105dce4c6e928fcb11cde56854', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:06.604750 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d4ecf857-385d-4556-a883-5595cf23a28c tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 200 len: 1312 microversion: 2.1 time: 0.178405 Sep 08 14:30:06.605483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1435/2585] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:06 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 1312 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:06.643329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6edecef9-707c-4a98-9caa-a1414caf1186 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:06.643658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6edecef9-707c-4a98-9caa-a1414caf1186 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:06.664313 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6edecef9-707c-4a98-9caa-a1414caf1186 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:06.664820 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6edecef9-707c-4a98-9caa-a1414caf1186 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:06.670081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6edecef9-707c-4a98-9caa-a1414caf1186 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:06.670972 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6edecef9-707c-4a98-9caa-a1414caf1186 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:06.671456 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6edecef9-707c-4a98-9caa-a1414caf1186 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:06.672049 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6edecef9-707c-4a98-9caa-a1414caf1186 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:06.948601 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6edecef9-707c-4a98-9caa-a1414caf1186 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:06.962322 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6edecef9-707c-4a98-9caa-a1414caf1186 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1366 microversion: 2.1 time: 0.353947 Sep 08 14:30:06.963004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1436/2586] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:06 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1366 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:06.969826 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-77d955f3-580b-4304-b66d-0fcecccb78c1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:06.970734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-77d955f3-580b-4304-b66d-0fcecccb78c1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:06.982447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77d955f3-580b-4304-b66d-0fcecccb78c1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:06.984897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77d955f3-580b-4304-b66d-0fcecccb78c1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:06.985883 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77d955f3-580b-4304-b66d-0fcecccb78c1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:07.005124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77d955f3-580b-4304-b66d-0fcecccb78c1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:07.005749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77d955f3-580b-4304-b66d-0fcecccb78c1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:07.006271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-77d955f3-580b-4304-b66d-0fcecccb78c1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:07.023870 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-b4eb3df8-38ad-4465-b25e-46a63104761c req-d73edf02-d5cb-45f8-8088-85943b8fb92f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bcef9a84-43d0-440b-804b-4a32893fc0d4", "name": "network-vif-plugged", "status": "completed", "tag": "c960c128-6e5a-4241-bfed-484d84bfbc84"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:30:07.050097 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-b4eb3df8-38ad-4465-b25e-46a63104761c req-d73edf02-d5cb-45f8-8088-85943b8fb92f service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:07.050625 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-b4eb3df8-38ad-4465-b25e-46a63104761c req-d73edf02-d5cb-45f8-8088-85943b8fb92f service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:07.051064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-b4eb3df8-38ad-4465-b25e-46a63104761c req-d73edf02-d5cb-45f8-8088-85943b8fb92f service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:07.066259 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-b4eb3df8-38ad-4465-b25e-46a63104761c req-d73edf02-d5cb-45f8-8088-85943b8fb92f service nova] Creating event network-vif-plugged:c960c128-6e5a-4241-bfed-484d84bfbc84 for instance bcef9a84-43d0-440b-804b-4a32893fc0d4 on hv-cinder-85646 Sep 08 14:30:07.072635 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-b4eb3df8-38ad-4465-b25e-46a63104761c req-d73edf02-d5cb-45f8-8088-85943b8fb92f service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.531702 Sep 08 14:30:07.073915 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1151/2587] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:30:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:30:07.081630 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e6710c7d-f1c7-4576-91f7-cf8cc2ca6ae5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:07.082363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e6710c7d-f1c7-4576-91f7-cf8cc2ca6ae5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:07.088696 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6710c7d-f1c7-4576-91f7-cf8cc2ca6ae5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:07.091850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6710c7d-f1c7-4576-91f7-cf8cc2ca6ae5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:07.091850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6710c7d-f1c7-4576-91f7-cf8cc2ca6ae5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:07.091850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6710c7d-f1c7-4576-91f7-cf8cc2ca6ae5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:07.092159 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6710c7d-f1c7-4576-91f7-cf8cc2ca6ae5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:07.092440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6710c7d-f1c7-4576-91f7-cf8cc2ca6ae5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:07.487626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e6710c7d-f1c7-4576-91f7-cf8cc2ca6ae5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:07.500990 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e6710c7d-f1c7-4576-91f7-cf8cc2ca6ae5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:07.500990 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e6710c7d-f1c7-4576-91f7-cf8cc2ca6ae5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:07.502282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e6710c7d-f1c7-4576-91f7-cf8cc2ca6ae5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1405 microversion: 2.20 time: 0.426366 Sep 08 14:30:07.503144 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1152/2588] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:30:07 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1405 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:30:07.583396 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-77d955f3-580b-4304-b66d-0fcecccb78c1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:07.595075 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-77d955f3-580b-4304-b66d-0fcecccb78c1 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 200 len: 1319 microversion: 2.1 time: 0.628747 Sep 08 14:30:07.608527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1437/2589] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:06 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 1319 bytes in 644 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:07.644252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-993f9365-e8af-4d0d-8fe9-ece768a1ab2a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:07.644868 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-993f9365-e8af-4d0d-8fe9-ece768a1ab2a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] [instance: 2707cec6-5b8c-41b5-8ae3-1b4c4de591af] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:07.652074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-993f9365-e8af-4d0d-8fe9-ece768a1ab2a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:07.652666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-993f9365-e8af-4d0d-8fe9-ece768a1ab2a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:07.653172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-993f9365-e8af-4d0d-8fe9-ece768a1ab2a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:07.653935 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-993f9365-e8af-4d0d-8fe9-ece768a1ab2a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:07.654494 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-993f9365-e8af-4d0d-8fe9-ece768a1ab2a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:07.654979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-993f9365-e8af-4d0d-8fe9-ece768a1ab2a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:07.669848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-993f9365-e8af-4d0d-8fe9-ece768a1ab2a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] HTTP exception thrown: Instance 2707cec6-5b8c-41b5-8ae3-1b4c4de591af could not be found. Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-993f9365-e8af-4d0d-8fe9-ece768a1ab2a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Request method failure captured: Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af HTTP/1.1 Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPpGZzMisIeN0rinm76oerWeI1e2n1ZCDvu1_QxGMaqly5J66d47JWYR-nVcthn5vbK6XFUxRx7ULj4gJs_JU089dQK1DqQD0zha4-7jxb_89w0QMqZgtACCW34zdnpCyIfB12dgyAWUWv38AgePo6JrTClIfdFoSLdR5OVWnj2kmY Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 1346d4105dce4c6e928fcb11cde56854 Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-188566618 Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:30:07.671544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:30:07.677026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1346d4105dce4c6e928fcb11cde56854"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:30:07.677026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-ServerRescueNegativeTestJSON-188566618 Sep 08 14:30:07.677026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 1346d4105dce4c6e928fcb11cde56854 Sep 08 14:30:07.677026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-188566618 Sep 08 14:30:07.677026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-ServerRescueNegativeTestJSON-188566618-project Sep 08 14:30:07.677026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:30:07.677026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:30:07.677026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 792808dcbf894aa98c671cf43ac21b8b Sep 08 14:30:07.677026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-ServerRescueNegativeTestJSON-188566618-project Sep 08 14:30:07.677026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:30:07.677026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 2707cec6-5b8c-41b5-8ae3-1b4c4de591af could not be found. Sep 08 14:30:07.677026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '2707cec6-5b8c-41b5-8ae3-1b4c4de591af'} Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 2707cec6-5b8c-41b5-8ae3-1b4c4de591af could not be found. Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:30:07.678058 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:30:07.679628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:30:07.679628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:30:07.679628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:30:07.679628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:30:07.679628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:30:07.679628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:30:07.679628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:30:07.679628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 2707cec6-5b8c-41b5-8ae3-1b4c4de591af could not be found. Sep 08 14:30:07.679628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:30:07.679628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:30:07.679628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:30:07.679628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:30:07.679628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:30:07.679628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:30:07.679628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:30:07.679628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:30:07.679628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 2707cec6-5b8c-41b5-8ae3-1b4c4de591af could not be found. Sep 08 14:30:07.679628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:30:07.680912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-993f9365-e8af-4d0d-8fe9-ece768a1ab2a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] Returning 404 to user: Instance 2707cec6-5b8c-41b5-8ae3-1b4c4de591af could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:30:07.680912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-993f9365-e8af-4d0d-8fe9-ece768a1ab2a tempest-ServerRescueNegativeTestJSON-188566618 tempest-ServerRescueNegativeTestJSON-188566618-project] 10.222.0.25 "GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af" status: 404 len: 111 microversion: 2.1 time: 0.032740 Sep 08 14:30:07.680912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1153/2590] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:07 2022] GET /compute/v2.1/servers/2707cec6-5b8c-41b5-8ae3-1b4c4de591af => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:30:07.987277 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3b73a9c1-0842-484b-8bb8-14d83b32cf6a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:07.987739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3b73a9c1-0842-484b-8bb8-14d83b32cf6a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:07.994376 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3b73a9c1-0842-484b-8bb8-14d83b32cf6a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:07.994895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3b73a9c1-0842-484b-8bb8-14d83b32cf6a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:07.995620 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3b73a9c1-0842-484b-8bb8-14d83b32cf6a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:07.996409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3b73a9c1-0842-484b-8bb8-14d83b32cf6a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:07.996883 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3b73a9c1-0842-484b-8bb8-14d83b32cf6a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:07.997652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3b73a9c1-0842-484b-8bb8-14d83b32cf6a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:08.430478 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3b73a9c1-0842-484b-8bb8-14d83b32cf6a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:08.438891 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3b73a9c1-0842-484b-8bb8-14d83b32cf6a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1366 microversion: 2.1 time: 0.455876 Sep 08 14:30:08.441008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1438/2591] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:07 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1366 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:08.532892 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8014b050-57cf-42ea-9cda-aec642fed827 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:08.533509 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8014b050-57cf-42ea-9cda-aec642fed827 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:08.541168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8014b050-57cf-42ea-9cda-aec642fed827 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:08.541655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8014b050-57cf-42ea-9cda-aec642fed827 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:08.544831 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8014b050-57cf-42ea-9cda-aec642fed827 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:08.545237 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8014b050-57cf-42ea-9cda-aec642fed827 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:08.545720 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8014b050-57cf-42ea-9cda-aec642fed827 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:08.546384 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8014b050-57cf-42ea-9cda-aec642fed827 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:08.624308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-37e77010-39e8-4474-8cb7-5b3381b8442b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:08.625345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-37e77010-39e8-4474-8cb7-5b3381b8442b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:08.632140 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-37e77010-39e8-4474-8cb7-5b3381b8442b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:08.632676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-37e77010-39e8-4474-8cb7-5b3381b8442b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:08.633065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-37e77010-39e8-4474-8cb7-5b3381b8442b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:08.633961 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-37e77010-39e8-4474-8cb7-5b3381b8442b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:08.634771 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-37e77010-39e8-4474-8cb7-5b3381b8442b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:08.635213 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-37e77010-39e8-4474-8cb7-5b3381b8442b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:08.714555 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8014b050-57cf-42ea-9cda-aec642fed827 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:08.728551 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8014b050-57cf-42ea-9cda-aec642fed827 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:08.728551 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8014b050-57cf-42ea-9cda-aec642fed827 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:08.728551 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8014b050-57cf-42ea-9cda-aec642fed827 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1405 microversion: 2.20 time: 0.198941 Sep 08 14:30:08.730224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1154/2592] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:30:08 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1405 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:30:08.787839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-9b869d67-b904-4e3e-bc8d-a964c0ae9432 req-155bace0-5011-447e-84f2-678681fb29ee service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6c129af4-e7f0-4743-85fd-5889547510f3", "tag": "968b521a-c5c0-4679-87a8-ddd5842563b7"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:30:08.797409 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-9b869d67-b904-4e3e-bc8d-a964c0ae9432 req-155bace0-5011-447e-84f2-678681fb29ee service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:08.797933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-9b869d67-b904-4e3e-bc8d-a964c0ae9432 req-155bace0-5011-447e-84f2-678681fb29ee service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:08.798397 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-9b869d67-b904-4e3e-bc8d-a964c0ae9432 req-155bace0-5011-447e-84f2-678681fb29ee service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:08.810832 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-9b869d67-b904-4e3e-bc8d-a964c0ae9432 req-155bace0-5011-447e-84f2-678681fb29ee service nova] Creating event network-changed:968b521a-c5c0-4679-87a8-ddd5842563b7 for instance 6c129af4-e7f0-4743-85fd-5889547510f3 on hv-cinder-85646 Sep 08 14:30:08.815970 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-9b869d67-b904-4e3e-bc8d-a964c0ae9432 req-155bace0-5011-447e-84f2-678681fb29ee service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031871 Sep 08 14:30:08.816724 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1155/2593] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:30:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:30:09.040803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-37e77010-39e8-4474-8cb7-5b3381b8442b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:09.062716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-37e77010-39e8-4474-8cb7-5b3381b8442b tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 200 len: 1319 microversion: 2.1 time: 0.447275 Sep 08 14:30:09.063729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1439/2594] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:08 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 1319 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:30:09.468020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f211f40b-55b2-4fc9-937a-e15bd5ffed63 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:09.469533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f211f40b-55b2-4fc9-937a-e15bd5ffed63 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:09.478481 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f211f40b-55b2-4fc9-937a-e15bd5ffed63 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:09.479859 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f211f40b-55b2-4fc9-937a-e15bd5ffed63 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:09.481305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f211f40b-55b2-4fc9-937a-e15bd5ffed63 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:09.482439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f211f40b-55b2-4fc9-937a-e15bd5ffed63 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:09.483089 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f211f40b-55b2-4fc9-937a-e15bd5ffed63 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:09.485748 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f211f40b-55b2-4fc9-937a-e15bd5ffed63 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:09.694544 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f211f40b-55b2-4fc9-937a-e15bd5ffed63 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:09.704545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f211f40b-55b2-4fc9-937a-e15bd5ffed63 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1366 microversion: 2.1 time: 0.243876 Sep 08 14:30:09.709405 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1156/2595] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:09 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1366 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:09.760390 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1c271f62-adb2-4f85-b905-1242dac7eaaf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:09.761473 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1c271f62-adb2-4f85-b905-1242dac7eaaf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:09.768795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1c271f62-adb2-4f85-b905-1242dac7eaaf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:09.769447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1c271f62-adb2-4f85-b905-1242dac7eaaf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:09.770213 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1c271f62-adb2-4f85-b905-1242dac7eaaf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:09.770998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1c271f62-adb2-4f85-b905-1242dac7eaaf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:09.771838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1c271f62-adb2-4f85-b905-1242dac7eaaf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:09.772343 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1c271f62-adb2-4f85-b905-1242dac7eaaf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:10.087297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5d10a984-6ee4-407c-993a-6448276f78b2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:10.087968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5d10a984-6ee4-407c-993a-6448276f78b2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:10.096242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d10a984-6ee4-407c-993a-6448276f78b2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:10.096926 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d10a984-6ee4-407c-993a-6448276f78b2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:10.097531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d10a984-6ee4-407c-993a-6448276f78b2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:10.098297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d10a984-6ee4-407c-993a-6448276f78b2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:10.098752 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d10a984-6ee4-407c-993a-6448276f78b2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:10.099219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5d10a984-6ee4-407c-993a-6448276f78b2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:10.275720 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5d10a984-6ee4-407c-993a-6448276f78b2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:10.287499 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5d10a984-6ee4-407c-993a-6448276f78b2 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 200 len: 1319 microversion: 2.1 time: 0.205178 Sep 08 14:30:10.288510 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1157/2596] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:10 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 1319 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:30:10.340832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1c271f62-adb2-4f85-b905-1242dac7eaaf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:10.356020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1c271f62-adb2-4f85-b905-1242dac7eaaf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:10.359356 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1c271f62-adb2-4f85-b905-1242dac7eaaf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:10.360801 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1c271f62-adb2-4f85-b905-1242dac7eaaf tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1405 microversion: 2.20 time: 0.605020 Sep 08 14:30:10.362109 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1440/2597] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:30:09 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1405 bytes in 607 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:30:10.567677 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-39de3cb4-3768-4762-918a-d9ccc1b40734 req-4f37e25d-e172-42c8-9805-f2047061f4da service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b0741665-d24b-4c17-93c7-90bcfff633b9", "name": "network-vif-plugged", "status": "completed", "tag": "67fe80b7-8ae7-46be-8809-bd4958623c19"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:30:10.578102 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-39de3cb4-3768-4762-918a-d9ccc1b40734 req-4f37e25d-e172-42c8-9805-f2047061f4da service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:10.578605 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-39de3cb4-3768-4762-918a-d9ccc1b40734 req-4f37e25d-e172-42c8-9805-f2047061f4da service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:10.579041 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-39de3cb4-3768-4762-918a-d9ccc1b40734 req-4f37e25d-e172-42c8-9805-f2047061f4da service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:10.590707 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-39de3cb4-3768-4762-918a-d9ccc1b40734 req-4f37e25d-e172-42c8-9805-f2047061f4da service nova] Creating event network-vif-plugged:67fe80b7-8ae7-46be-8809-bd4958623c19 for instance b0741665-d24b-4c17-93c7-90bcfff633b9 on hv-cinder-85646 Sep 08 14:30:10.601638 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-39de3cb4-3768-4762-918a-d9ccc1b40734 req-4f37e25d-e172-42c8-9805-f2047061f4da service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.039280 Sep 08 14:30:10.602443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1158/2598] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:30:10 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:30:10.731631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-797aea86-6eea-4a8b-aa33-8f6c4d51d384 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:10.735245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-797aea86-6eea-4a8b-aa33-8f6c4d51d384 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:10.739020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-797aea86-6eea-4a8b-aa33-8f6c4d51d384 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:10.739700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-797aea86-6eea-4a8b-aa33-8f6c4d51d384 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:10.740144 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-797aea86-6eea-4a8b-aa33-8f6c4d51d384 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:10.740854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-797aea86-6eea-4a8b-aa33-8f6c4d51d384 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:10.741307 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-797aea86-6eea-4a8b-aa33-8f6c4d51d384 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:10.741734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-797aea86-6eea-4a8b-aa33-8f6c4d51d384 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:11.017243 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-797aea86-6eea-4a8b-aa33-8f6c4d51d384 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:11.028526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-797aea86-6eea-4a8b-aa33-8f6c4d51d384 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1556 microversion: 2.1 time: 0.301497 Sep 08 14:30:11.047274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1441/2599] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:10 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1556 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:11.056948 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cd1526eb-2f53-4831-a44e-7037813b2a9a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:11.057893 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cd1526eb-2f53-4831-a44e-7037813b2a9a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:11.062857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=94,nova_api:INSERT=6 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:11.068332 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd1526eb-2f53-4831-a44e-7037813b2a9a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:11.068825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd1526eb-2f53-4831-a44e-7037813b2a9a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:11.069262 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd1526eb-2f53-4831-a44e-7037813b2a9a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:11.069979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd1526eb-2f53-4831-a44e-7037813b2a9a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:11.070497 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd1526eb-2f53-4831-a44e-7037813b2a9a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:11.070928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cd1526eb-2f53-4831-a44e-7037813b2a9a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:11.172639 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cd1526eb-2f53-4831-a44e-7037813b2a9a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.123139 Sep 08 14:30:11.173244 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1159/2600] 10.222.0.25 () {58 vars in 1275 bytes} [Thu Sep 8 14:30:11 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4/os-interface => generated 285 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:30:11.313887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-50e6f27b-6a0d-446a-9644-b018cfb82330 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:11.314763 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-50e6f27b-6a0d-446a-9644-b018cfb82330 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:11.324053 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50e6f27b-6a0d-446a-9644-b018cfb82330 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:11.325335 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50e6f27b-6a0d-446a-9644-b018cfb82330 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:11.325999 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50e6f27b-6a0d-446a-9644-b018cfb82330 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:11.327026 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50e6f27b-6a0d-446a-9644-b018cfb82330 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:11.327871 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50e6f27b-6a0d-446a-9644-b018cfb82330 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:11.328590 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-50e6f27b-6a0d-446a-9644-b018cfb82330 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:11.391403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=2 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:11.396197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c48a49e8-d966-472a-8a37-220c714d2227 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:11.398007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c48a49e8-d966-472a-8a37-220c714d2227 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:11.407156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c48a49e8-d966-472a-8a37-220c714d2227 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:11.408066 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c48a49e8-d966-472a-8a37-220c714d2227 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:11.408599 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c48a49e8-d966-472a-8a37-220c714d2227 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:11.409465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c48a49e8-d966-472a-8a37-220c714d2227 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:11.410024 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c48a49e8-d966-472a-8a37-220c714d2227 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:11.410625 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c48a49e8-d966-472a-8a37-220c714d2227 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:11.568610 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c48a49e8-d966-472a-8a37-220c714d2227 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:11.580243 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c48a49e8-d966-472a-8a37-220c714d2227 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:11.581857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c48a49e8-d966-472a-8a37-220c714d2227 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:11.583385 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c48a49e8-d966-472a-8a37-220c714d2227 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1405 microversion: 2.20 time: 0.192835 Sep 08 14:30:11.584046 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1160/2601] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:30:11 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1405 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 14:30:11.822160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-50e6f27b-6a0d-446a-9644-b018cfb82330 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:11.832730 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-50e6f27b-6a0d-446a-9644-b018cfb82330 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 200 len: 1319 microversion: 2.1 time: 0.523608 Sep 08 14:30:11.833597 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1442/2602] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:11 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 1319 bytes in 525 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:12.614420 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=22 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:12.619303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d6a5a4a6-d581-4517-b3ff-01746e31fe54 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:12.619903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d6a5a4a6-d581-4517-b3ff-01746e31fe54 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:12.625924 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d6a5a4a6-d581-4517-b3ff-01746e31fe54 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:12.626425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d6a5a4a6-d581-4517-b3ff-01746e31fe54 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:12.626854 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d6a5a4a6-d581-4517-b3ff-01746e31fe54 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:12.627533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d6a5a4a6-d581-4517-b3ff-01746e31fe54 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:12.627973 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d6a5a4a6-d581-4517-b3ff-01746e31fe54 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:12.628384 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d6a5a4a6-d581-4517-b3ff-01746e31fe54 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:12.776932 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d6a5a4a6-d581-4517-b3ff-01746e31fe54 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:12.786469 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d6a5a4a6-d581-4517-b3ff-01746e31fe54 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:12.787909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d6a5a4a6-d581-4517-b3ff-01746e31fe54 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:12.789028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d6a5a4a6-d581-4517-b3ff-01746e31fe54 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1405 microversion: 2.20 time: 0.176193 Sep 08 14:30:12.789594 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1161/2603] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:30:12 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1405 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:30:12.861762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8d1ddb04-d5a6-4273-845d-6581ed90f0e4 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:12.862422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8d1ddb04-d5a6-4273-845d-6581ed90f0e4 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:12.870168 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8d1ddb04-d5a6-4273-845d-6581ed90f0e4 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:12.875250 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8d1ddb04-d5a6-4273-845d-6581ed90f0e4 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:12.875998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8d1ddb04-d5a6-4273-845d-6581ed90f0e4 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:12.876747 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8d1ddb04-d5a6-4273-845d-6581ed90f0e4 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:12.877752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8d1ddb04-d5a6-4273-845d-6581ed90f0e4 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:12.877752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8d1ddb04-d5a6-4273-845d-6581ed90f0e4 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:13.289859 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8d1ddb04-d5a6-4273-845d-6581ed90f0e4 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:13.305470 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8d1ddb04-d5a6-4273-845d-6581ed90f0e4 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 200 len: 1319 microversion: 2.1 time: 0.448401 Sep 08 14:30:13.306072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1443/2604] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:12 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 1319 bytes in 450 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:13.637513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-3e33dc4b-02c1-4cd1-afc6-677c021b5099 req-19ff0c65-618d-4967-861d-3d54ac9892ad service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bcef9a84-43d0-440b-804b-4a32893fc0d4", "tag": "c960c128-6e5a-4241-bfed-484d84bfbc84"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:30:13.648599 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-3e33dc4b-02c1-4cd1-afc6-677c021b5099 req-19ff0c65-618d-4967-861d-3d54ac9892ad service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:13.649265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-3e33dc4b-02c1-4cd1-afc6-677c021b5099 req-19ff0c65-618d-4967-861d-3d54ac9892ad service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:13.649529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-3e33dc4b-02c1-4cd1-afc6-677c021b5099 req-19ff0c65-618d-4967-861d-3d54ac9892ad service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:13.664330 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-3e33dc4b-02c1-4cd1-afc6-677c021b5099 req-19ff0c65-618d-4967-861d-3d54ac9892ad service nova] Creating event network-changed:c960c128-6e5a-4241-bfed-484d84bfbc84 for instance bcef9a84-43d0-440b-804b-4a32893fc0d4 on hv-cinder-85646 Sep 08 14:30:13.670340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-3e33dc4b-02c1-4cd1-afc6-677c021b5099 req-19ff0c65-618d-4967-861d-3d54ac9892ad service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036855 Sep 08 14:30:13.671820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1162/2605] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:30:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:30:13.819215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ba73dc56-8b35-4fb1-9c75-2452d8292290 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:13.819831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ba73dc56-8b35-4fb1-9c75-2452d8292290 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:13.831103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba73dc56-8b35-4fb1-9c75-2452d8292290 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:13.832399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba73dc56-8b35-4fb1-9c75-2452d8292290 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:13.833085 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba73dc56-8b35-4fb1-9c75-2452d8292290 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:13.834002 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba73dc56-8b35-4fb1-9c75-2452d8292290 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:13.834676 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba73dc56-8b35-4fb1-9c75-2452d8292290 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:13.835286 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ba73dc56-8b35-4fb1-9c75-2452d8292290 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:14.174428 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ba73dc56-8b35-4fb1-9c75-2452d8292290 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:14.192671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ba73dc56-8b35-4fb1-9c75-2452d8292290 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:14.202051 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ba73dc56-8b35-4fb1-9c75-2452d8292290 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:14.205689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ba73dc56-8b35-4fb1-9c75-2452d8292290 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1595 microversion: 2.20 time: 0.390319 Sep 08 14:30:14.207498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1444/2606] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:30:13 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1595 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:30:14.233962 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a707ba6f-1982-410e-976f-190194fd9c39 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:14.235571 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a707ba6f-1982-410e-976f-190194fd9c39 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:14.242013 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a707ba6f-1982-410e-976f-190194fd9c39 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:14.242533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a707ba6f-1982-410e-976f-190194fd9c39 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:14.242955 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a707ba6f-1982-410e-976f-190194fd9c39 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:14.243681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a707ba6f-1982-410e-976f-190194fd9c39 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:14.244124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a707ba6f-1982-410e-976f-190194fd9c39 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:14.244539 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a707ba6f-1982-410e-976f-190194fd9c39 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:14.333394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-792e334d-02de-4da5-95de-e7ac37bd113e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:14.334066 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-792e334d-02de-4da5-95de-e7ac37bd113e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:14.345528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-792e334d-02de-4da5-95de-e7ac37bd113e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:14.345528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-792e334d-02de-4da5-95de-e7ac37bd113e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:14.345528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-792e334d-02de-4da5-95de-e7ac37bd113e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:14.347960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-792e334d-02de-4da5-95de-e7ac37bd113e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:14.348590 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-792e334d-02de-4da5-95de-e7ac37bd113e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:14.349087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-792e334d-02de-4da5-95de-e7ac37bd113e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:14.353872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a707ba6f-1982-410e-976f-190194fd9c39 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.124574 Sep 08 14:30:14.354774 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1163/2607] 10.222.0.25 () {60 vars in 1316 bytes} [Thu Sep 8 14:30:14 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9/os-interface => generated 284 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:30:14.695918 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-792e334d-02de-4da5-95de-e7ac37bd113e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:14.716625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-792e334d-02de-4da5-95de-e7ac37bd113e tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 200 len: 1319 microversion: 2.1 time: 0.387287 Sep 08 14:30:14.717848 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1445/2608] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:14 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 1319 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:30:14.852455 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-5e224c7f-e487-4174-92e3-612369c97055 req-5d3a243d-8151-4189-beb4-14e9d7acd10d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6c129af4-e7f0-4743-85fd-5889547510f3", "name": "network-vif-plugged", "status": "completed", "tag": "968b521a-c5c0-4679-87a8-ddd5842563b7"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:30:14.869973 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-5e224c7f-e487-4174-92e3-612369c97055 req-5d3a243d-8151-4189-beb4-14e9d7acd10d service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:14.870483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-5e224c7f-e487-4174-92e3-612369c97055 req-5d3a243d-8151-4189-beb4-14e9d7acd10d service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:14.871446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-5e224c7f-e487-4174-92e3-612369c97055 req-5d3a243d-8151-4189-beb4-14e9d7acd10d service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:14.892579 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-5e224c7f-e487-4174-92e3-612369c97055 req-5d3a243d-8151-4189-beb4-14e9d7acd10d service nova] Creating event network-vif-plugged:968b521a-c5c0-4679-87a8-ddd5842563b7 for instance 6c129af4-e7f0-4743-85fd-5889547510f3 on hv-cinder-85646 Sep 08 14:30:14.899884 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-5e224c7f-e487-4174-92e3-612369c97055 req-5d3a243d-8151-4189-beb4-14e9d7acd10d service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052143 Sep 08 14:30:14.900833 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1164/2609] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:30:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:30:15.713043 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-c01c9315-857e-42c7-8f1a-463ed42506ff req-a512df2c-cff8-465e-83ba-e5b27b95621e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "b0741665-d24b-4c17-93c7-90bcfff633b9", "tag": "67fe80b7-8ae7-46be-8809-bd4958623c19"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:30:15.724099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-c01c9315-857e-42c7-8f1a-463ed42506ff req-a512df2c-cff8-465e-83ba-e5b27b95621e service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:15.724584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-c01c9315-857e-42c7-8f1a-463ed42506ff req-a512df2c-cff8-465e-83ba-e5b27b95621e service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:15.725010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-c01c9315-857e-42c7-8f1a-463ed42506ff req-a512df2c-cff8-465e-83ba-e5b27b95621e service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:15.738431 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-c01c9315-857e-42c7-8f1a-463ed42506ff req-a512df2c-cff8-465e-83ba-e5b27b95621e service nova] Creating event network-changed:67fe80b7-8ae7-46be-8809-bd4958623c19 for instance b0741665-d24b-4c17-93c7-90bcfff633b9 on hv-cinder-85646 Sep 08 14:30:15.743698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-c01c9315-857e-42c7-8f1a-463ed42506ff req-a512df2c-cff8-465e-83ba-e5b27b95621e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034274 Sep 08 14:30:15.744384 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1446/2610] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:30:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:30:15.745336 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-16f343fc-7849-4978-8ed3-5570938d9bed tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:15.746301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-16f343fc-7849-4978-8ed3-5570938d9bed tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:15.757095 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-16f343fc-7849-4978-8ed3-5570938d9bed tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:15.759448 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-16f343fc-7849-4978-8ed3-5570938d9bed tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:15.760308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-16f343fc-7849-4978-8ed3-5570938d9bed tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:15.761361 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-16f343fc-7849-4978-8ed3-5570938d9bed tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:15.762878 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-16f343fc-7849-4978-8ed3-5570938d9bed tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:15.763383 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-16f343fc-7849-4978-8ed3-5570938d9bed tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:15.941445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-16f343fc-7849-4978-8ed3-5570938d9bed tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:15.954455 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-16f343fc-7849-4978-8ed3-5570938d9bed tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 200 len: 1319 microversion: 2.1 time: 0.210096 Sep 08 14:30:15.962682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1165/2611] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:15 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 1319 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:16.975705 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4ed81adf-bd2b-47b5-a8db-217ed8650974 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:16.976404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4ed81adf-bd2b-47b5-a8db-217ed8650974 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:16.983181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ed81adf-bd2b-47b5-a8db-217ed8650974 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:16.983666 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ed81adf-bd2b-47b5-a8db-217ed8650974 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:16.984104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ed81adf-bd2b-47b5-a8db-217ed8650974 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:16.984795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ed81adf-bd2b-47b5-a8db-217ed8650974 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:16.985252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ed81adf-bd2b-47b5-a8db-217ed8650974 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:16.985693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ed81adf-bd2b-47b5-a8db-217ed8650974 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:17.537913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4ed81adf-bd2b-47b5-a8db-217ed8650974 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:17.545957 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4ed81adf-bd2b-47b5-a8db-217ed8650974 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 200 len: 1319 microversion: 2.1 time: 0.575031 Sep 08 14:30:17.554004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1447/2612] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:16 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 1319 bytes in 577 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:18.568949 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-74c22ef9-53d3-4dea-8e3d-797eaa71913c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:18.569763 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-74c22ef9-53d3-4dea-8e3d-797eaa71913c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:18.576255 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74c22ef9-53d3-4dea-8e3d-797eaa71913c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:18.576727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74c22ef9-53d3-4dea-8e3d-797eaa71913c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:18.577152 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74c22ef9-53d3-4dea-8e3d-797eaa71913c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:18.577847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74c22ef9-53d3-4dea-8e3d-797eaa71913c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:18.578362 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74c22ef9-53d3-4dea-8e3d-797eaa71913c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:18.578777 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-74c22ef9-53d3-4dea-8e3d-797eaa71913c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:18.728315 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-74c22ef9-53d3-4dea-8e3d-797eaa71913c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:18.736879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-74c22ef9-53d3-4dea-8e3d-797eaa71913c tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 200 len: 1506 microversion: 2.1 time: 0.171869 Sep 08 14:30:18.737781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1166/2613] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:18 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 1506 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:18.759775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1567f8e9-0ab6-41f2-913b-db60144ee1b6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a2a8915a-f50a-4a19-a782-4c5b781c61ab", "device": "/dev/sdb"}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:30:18.764277 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1567f8e9-0ab6-41f2-913b-db60144ee1b6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:18.771388 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1567f8e9-0ab6-41f2-913b-db60144ee1b6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:18.772207 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1567f8e9-0ab6-41f2-913b-db60144ee1b6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:18.772650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1567f8e9-0ab6-41f2-913b-db60144ee1b6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:18.773972 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1567f8e9-0ab6-41f2-913b-db60144ee1b6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:18.774857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1567f8e9-0ab6-41f2-913b-db60144ee1b6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:18.775523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1567f8e9-0ab6-41f2-913b-db60144ee1b6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:19.616132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1567f8e9-0ab6-41f2-913b-db60144ee1b6 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "POST /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.860409 Sep 08 14:30:19.617331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1448/2614] 10.222.0.25 () {60 vars in 1323 bytes} [Thu Sep 8 14:30:18 2022] POST /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3/os-volume_attachments => generated 194 bytes in 862 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:30:23.060280 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-25e7b945-451e-4387-85c2-2bcd6f50f06f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Action: 'create', calling method: .version_select at 0x7fce020c7f70>, body: {"keypair": {"name": "tempest-keypair-338464191", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNCpjm4xwl85wKf/bdL7LY6DHNwpKsMH/tYnO9nGJwvQPmhgP13CQeH8zFRnsDkLxzRuFvfglt3XW+YRRtnN+ZylBoobrwhETNeWoqSHr+KiF+6lxXiTsUlwoVXfMLZBhQ=="}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:30:23.080142 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-25e7b945-451e-4387-85c2-2bcd6f50f06f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Getting quotas for project b21f666643b74b5cba0b7597ae1aa0cd. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 14:30:23.084316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-25e7b945-451e-4387-85c2-2bcd6f50f06f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Getting quotas for user 357ebb86fd664e08b15bf440e2a61560 and project b21f666643b74b5cba0b7597ae1aa0cd. Resources: {'key_pairs'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 14:30:23.213502 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-25e7b945-451e-4387-85c2-2bcd6f50f06f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.298079 Sep 08 14:30:23.214473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1167/2615] 10.222.0.25 () {60 vars in 1159 bytes} [Thu Sep 8 14:30:22 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:30:25.566455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ad174fcb-3093-4944-ac75-de290213d572 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-458309787", "imageRef": "20b30425-92e5-4d78-8502-45557526ea49", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8c16dc28-690d-4d7e-aeb8-ee937ab86528"}], "security_groups": [{"name": "tempest-securitygroup--166302987"}], "key_name": "tempest-keypair-338464191", "user_data": "}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:30:25.575567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:INSERT=2,nova_api:SELECT=93,nova_api:UPDATE=2 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:25.701899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.network.neutron [None req-ad174fcb-3093-4944-ac75-de290213d572 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] validate_networks() for [('8c16dc28-690d-4d7e-aeb8-ee937ab86528', None, None, None, None, None)] {{(pid=127601) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 14:30:26.102110 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-ad174fcb-3093-4944-ac75-de290213d572 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Flavor limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 14:30:26.119782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-ad174fcb-3093-4944-ac75-de290213d572 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Image limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 14:30:26.119782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-ad174fcb-3093-4944-ac75-de290213d572 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Flavor pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 14:30:26.119782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-ad174fcb-3093-4944-ac75-de290213d572 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Image pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 14:30:26.119782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-ad174fcb-3093-4944-ac75-de290213d572 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 14:30:26.119782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad174fcb-3093-4944-ac75-de290213d572 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:26.120263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad174fcb-3093-4944-ac75-de290213d572 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:26.120263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad174fcb-3093-4944-ac75-de290213d572 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:26.120263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad174fcb-3093-4944-ac75-de290213d572 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:26.120263 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad174fcb-3093-4944-ac75-de290213d572 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:26.121259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ad174fcb-3093-4944-ac75-de290213d572 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:26.134050 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-ad174fcb-3093-4944-ac75-de290213d572 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Getting quotas for project b21f666643b74b5cba0b7597ae1aa0cd. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 14:30:26.139277 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-ad174fcb-3093-4944-ac75-de290213d572 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Getting quotas for user 357ebb86fd664e08b15bf440e2a61560 and project b21f666643b74b5cba0b7597ae1aa0cd. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 14:30:26.150845 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ad174fcb-3093-4944-ac75-de290213d572 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Going to run 1 instances... {{(pid=127601) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 14:30:26.153279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ad174fcb-3093-4944-ac75-de290213d572 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='20b30425-92e5-4d78-8502-45557526ea49',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127601) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 14:30:26.287786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ad174fcb-3093-4944-ac75-de290213d572 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 0.726735 Sep 08 14:30:26.288810 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1449/2616] 10.222.0.25 () {60 vars in 1147 bytes} [Thu Sep 8 14:30:25 2022] POST /compute/v2.1/servers => generated 401 bytes in 729 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 08 14:30:26.301301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6aa6ff30-b736-4589-9077-2d2ded559ad4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:26.302451 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6aa6ff30-b736-4589-9077-2d2ded559ad4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:26.378939 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6aa6ff30-b736-4589-9077-2d2ded559ad4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:26.386929 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6aa6ff30-b736-4589-9077-2d2ded559ad4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1236 microversion: 2.1 time: 0.090368 Sep 08 14:30:26.388110 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1168/2617] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:26 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1236 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:27.410879 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8136bb16-91c7-44f8-9b17-7f87ed459b89 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:27.412042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8136bb16-91c7-44f8-9b17-7f87ed459b89 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:27.418750 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8136bb16-91c7-44f8-9b17-7f87ed459b89 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:27.419494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8136bb16-91c7-44f8-9b17-7f87ed459b89 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:27.420118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8136bb16-91c7-44f8-9b17-7f87ed459b89 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:27.421006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8136bb16-91c7-44f8-9b17-7f87ed459b89 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:27.421468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8136bb16-91c7-44f8-9b17-7f87ed459b89 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:27.421935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8136bb16-91c7-44f8-9b17-7f87ed459b89 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:27.437760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:27.485207 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8136bb16-91c7-44f8-9b17-7f87ed459b89 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:27.493928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8136bb16-91c7-44f8-9b17-7f87ed459b89 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1240 microversion: 2.1 time: 0.087338 Sep 08 14:30:27.494526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1450/2618] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:27 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1240 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:28.518279 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2940f90b-db29-47bc-b411-50a2b7861558 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:28.518905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2940f90b-db29-47bc-b411-50a2b7861558 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:28.525900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2940f90b-db29-47bc-b411-50a2b7861558 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:28.526440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2940f90b-db29-47bc-b411-50a2b7861558 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:28.526864 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2940f90b-db29-47bc-b411-50a2b7861558 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:28.527549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2940f90b-db29-47bc-b411-50a2b7861558 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:28.528004 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2940f90b-db29-47bc-b411-50a2b7861558 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:28.528425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2940f90b-db29-47bc-b411-50a2b7861558 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:28.592216 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-2940f90b-db29-47bc-b411-50a2b7861558 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:28.600122 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2940f90b-db29-47bc-b411-50a2b7861558 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1288 microversion: 2.1 time: 0.086508 Sep 08 14:30:28.600690 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1169/2619] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:28 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1288 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:29.631064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0a8b4822-26f2-49fa-a8d9-3f24e1214df4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:29.631666 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0a8b4822-26f2-49fa-a8d9-3f24e1214df4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:29.639009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0a8b4822-26f2-49fa-a8d9-3f24e1214df4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:29.639501 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0a8b4822-26f2-49fa-a8d9-3f24e1214df4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:29.640021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0a8b4822-26f2-49fa-a8d9-3f24e1214df4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:29.640856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0a8b4822-26f2-49fa-a8d9-3f24e1214df4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:29.641167 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0a8b4822-26f2-49fa-a8d9-3f24e1214df4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:29.641593 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0a8b4822-26f2-49fa-a8d9-3f24e1214df4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:29.778047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0a8b4822-26f2-49fa-a8d9-3f24e1214df4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:29.786117 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0a8b4822-26f2-49fa-a8d9-3f24e1214df4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1361 microversion: 2.1 time: 0.159870 Sep 08 14:30:29.786707 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1451/2620] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:29 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1361 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:30.809242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b172b1f5-4b3d-4b30-a127-2b62979c755b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:30.809798 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b172b1f5-4b3d-4b30-a127-2b62979c755b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:30.816304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b172b1f5-4b3d-4b30-a127-2b62979c755b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:30.816784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b172b1f5-4b3d-4b30-a127-2b62979c755b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:30.817216 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b172b1f5-4b3d-4b30-a127-2b62979c755b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:30.817902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b172b1f5-4b3d-4b30-a127-2b62979c755b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:30.818383 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b172b1f5-4b3d-4b30-a127-2b62979c755b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:30.818804 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b172b1f5-4b3d-4b30-a127-2b62979c755b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:30.950459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b172b1f5-4b3d-4b30-a127-2b62979c755b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:30.958528 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b172b1f5-4b3d-4b30-a127-2b62979c755b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1361 microversion: 2.1 time: 0.153244 Sep 08 14:30:30.959100 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1170/2621] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:30 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1361 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:31.982443 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-15f770a5-cc3b-4605-8404-779e77bcd0e8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:31.983046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-15f770a5-cc3b-4605-8404-779e77bcd0e8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:31.989564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-15f770a5-cc3b-4605-8404-779e77bcd0e8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:31.990091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-15f770a5-cc3b-4605-8404-779e77bcd0e8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:31.990557 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-15f770a5-cc3b-4605-8404-779e77bcd0e8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:31.991229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-15f770a5-cc3b-4605-8404-779e77bcd0e8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:31.991685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-15f770a5-cc3b-4605-8404-779e77bcd0e8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:31.992101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-15f770a5-cc3b-4605-8404-779e77bcd0e8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:32.425516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-15f770a5-cc3b-4605-8404-779e77bcd0e8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:32.433068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-15f770a5-cc3b-4605-8404-779e77bcd0e8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1361 microversion: 2.1 time: 0.455103 Sep 08 14:30:32.433635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1452/2622] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:31 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1361 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:33.103057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-f470aca3-045c-4a5f-98e4-636831f53138 req-d2ba5621-b00b-4850-9c5b-a2b00eaf5d5d service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c09b905a-0795-4dc5-a413-007f3951e49f", "tag": "c06eb27d-e611-488c-80be-0549fc42ecef"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:30:33.114033 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-f470aca3-045c-4a5f-98e4-636831f53138 req-d2ba5621-b00b-4850-9c5b-a2b00eaf5d5d service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:33.114612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-f470aca3-045c-4a5f-98e4-636831f53138 req-d2ba5621-b00b-4850-9c5b-a2b00eaf5d5d service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:33.115052 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-f470aca3-045c-4a5f-98e4-636831f53138 req-d2ba5621-b00b-4850-9c5b-a2b00eaf5d5d service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:33.126816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-f470aca3-045c-4a5f-98e4-636831f53138 req-d2ba5621-b00b-4850-9c5b-a2b00eaf5d5d service nova] Creating event network-changed:c06eb27d-e611-488c-80be-0549fc42ecef for instance c09b905a-0795-4dc5-a413-007f3951e49f on hv-cinder-85646 Sep 08 14:30:33.131777 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-f470aca3-045c-4a5f-98e4-636831f53138 req-d2ba5621-b00b-4850-9c5b-a2b00eaf5d5d service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032533 Sep 08 14:30:33.132385 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1171/2623] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:30:33 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:30:33.456111 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0ca5c792-0094-471c-a1e2-b69b372f11ea tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:33.456831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0ca5c792-0094-471c-a1e2-b69b372f11ea tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:33.463659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ca5c792-0094-471c-a1e2-b69b372f11ea tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:33.464142 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ca5c792-0094-471c-a1e2-b69b372f11ea tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:33.465383 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ca5c792-0094-471c-a1e2-b69b372f11ea tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:33.466567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ca5c792-0094-471c-a1e2-b69b372f11ea tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:33.467293 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ca5c792-0094-471c-a1e2-b69b372f11ea tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:33.467962 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0ca5c792-0094-471c-a1e2-b69b372f11ea tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:33.637758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0ca5c792-0094-471c-a1e2-b69b372f11ea tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:33.645588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0ca5c792-0094-471c-a1e2-b69b372f11ea tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1361 microversion: 2.1 time: 0.193401 Sep 08 14:30:33.646201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1453/2624] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:33 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1361 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:34.670426 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c48e8707-b02c-4a18-949b-0a6e3f57ecf6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:34.671123 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c48e8707-b02c-4a18-949b-0a6e3f57ecf6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:34.677343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c48e8707-b02c-4a18-949b-0a6e3f57ecf6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:34.677823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c48e8707-b02c-4a18-949b-0a6e3f57ecf6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:34.678294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c48e8707-b02c-4a18-949b-0a6e3f57ecf6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:34.678978 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c48e8707-b02c-4a18-949b-0a6e3f57ecf6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:34.679428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c48e8707-b02c-4a18-949b-0a6e3f57ecf6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:34.679852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c48e8707-b02c-4a18-949b-0a6e3f57ecf6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:34.810408 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c48e8707-b02c-4a18-949b-0a6e3f57ecf6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:34.818166 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c48e8707-b02c-4a18-949b-0a6e3f57ecf6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1361 microversion: 2.1 time: 0.152943 Sep 08 14:30:34.818727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1172/2625] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:34 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1361 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:35.841197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c84b85f4-9d62-4d7d-86c5-27213f56d0e9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:35.841803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c84b85f4-9d62-4d7d-86c5-27213f56d0e9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:35.848755 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c84b85f4-9d62-4d7d-86c5-27213f56d0e9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:35.849249 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c84b85f4-9d62-4d7d-86c5-27213f56d0e9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:35.849692 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c84b85f4-9d62-4d7d-86c5-27213f56d0e9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:35.850451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c84b85f4-9d62-4d7d-86c5-27213f56d0e9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:35.850916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c84b85f4-9d62-4d7d-86c5-27213f56d0e9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:35.851344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c84b85f4-9d62-4d7d-86c5-27213f56d0e9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:35.978917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c84b85f4-9d62-4d7d-86c5-27213f56d0e9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:35.986410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c84b85f4-9d62-4d7d-86c5-27213f56d0e9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1361 microversion: 2.1 time: 0.149492 Sep 08 14:30:35.986975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1454/2626] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:35 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1361 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:37.008975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9c58bd7f-b08a-4a00-b569-b2866ce1a476 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:37.009582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9c58bd7f-b08a-4a00-b569-b2866ce1a476 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:37.012308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=6 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:37.017194 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9c58bd7f-b08a-4a00-b569-b2866ce1a476 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:37.017707 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9c58bd7f-b08a-4a00-b569-b2866ce1a476 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:37.018167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9c58bd7f-b08a-4a00-b569-b2866ce1a476 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:37.021627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9c58bd7f-b08a-4a00-b569-b2866ce1a476 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:37.021627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9c58bd7f-b08a-4a00-b569-b2866ce1a476 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:37.021627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9c58bd7f-b08a-4a00-b569-b2866ce1a476 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:37.023705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=95 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:37.147885 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9c58bd7f-b08a-4a00-b569-b2866ce1a476 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:37.156067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9c58bd7f-b08a-4a00-b569-b2866ce1a476 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1361 microversion: 2.1 time: 0.151083 Sep 08 14:30:37.156638 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1173/2627] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:37 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1361 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:38.178866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a4b116db-e91f-4f0c-acf6-e58b4e6724db tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:38.179469 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a4b116db-e91f-4f0c-acf6-e58b4e6724db tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:38.181339 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=2 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:38.182016 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=6 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:38.187821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4b116db-e91f-4f0c-acf6-e58b4e6724db tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:38.188312 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4b116db-e91f-4f0c-acf6-e58b4e6724db tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:38.191625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4b116db-e91f-4f0c-acf6-e58b4e6724db tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:38.191625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4b116db-e91f-4f0c-acf6-e58b4e6724db tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:38.191625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4b116db-e91f-4f0c-acf6-e58b4e6724db tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:38.191625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a4b116db-e91f-4f0c-acf6-e58b4e6724db tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:38.318936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a4b116db-e91f-4f0c-acf6-e58b4e6724db tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:38.326398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a4b116db-e91f-4f0c-acf6-e58b4e6724db tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1361 microversion: 2.1 time: 0.151341 Sep 08 14:30:38.326961 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1455/2628] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:38 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1361 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:39.348813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-73a5daf2-407a-4a39-83de-ee18e50b3d87 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:39.349457 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-73a5daf2-407a-4a39-83de-ee18e50b3d87 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:39.356550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73a5daf2-407a-4a39-83de-ee18e50b3d87 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:39.357025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73a5daf2-407a-4a39-83de-ee18e50b3d87 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:39.357460 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73a5daf2-407a-4a39-83de-ee18e50b3d87 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:39.358168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73a5daf2-407a-4a39-83de-ee18e50b3d87 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:39.358620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73a5daf2-407a-4a39-83de-ee18e50b3d87 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:39.359041 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-73a5daf2-407a-4a39-83de-ee18e50b3d87 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:39.491361 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-73a5daf2-407a-4a39-83de-ee18e50b3d87 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:39.498974 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-73a5daf2-407a-4a39-83de-ee18e50b3d87 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1361 microversion: 2.1 time: 0.154054 Sep 08 14:30:39.499536 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1174/2629] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:39 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1361 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:40.522211 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8b6c5036-5ebf-4018-9d90-4eb619fdd3a9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:40.522805 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8b6c5036-5ebf-4018-9d90-4eb619fdd3a9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:40.529136 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b6c5036-5ebf-4018-9d90-4eb619fdd3a9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:40.529623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b6c5036-5ebf-4018-9d90-4eb619fdd3a9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:40.530060 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b6c5036-5ebf-4018-9d90-4eb619fdd3a9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:40.530769 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b6c5036-5ebf-4018-9d90-4eb619fdd3a9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:40.531226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b6c5036-5ebf-4018-9d90-4eb619fdd3a9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:40.531684 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8b6c5036-5ebf-4018-9d90-4eb619fdd3a9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:40.668630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8b6c5036-5ebf-4018-9d90-4eb619fdd3a9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:40.676264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8b6c5036-5ebf-4018-9d90-4eb619fdd3a9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1361 microversion: 2.1 time: 0.157998 Sep 08 14:30:40.676837 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1456/2630] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:40 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1361 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:40.877758 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a53de3fe-266e-4b1b-b3a6-ec948f8ee4fa tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:40.878410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a53de3fe-266e-4b1b-b3a6-ec948f8ee4fa tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:40.884850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a53de3fe-266e-4b1b-b3a6-ec948f8ee4fa tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:40.885333 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a53de3fe-266e-4b1b-b3a6-ec948f8ee4fa tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:40.885776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a53de3fe-266e-4b1b-b3a6-ec948f8ee4fa tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:40.886542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a53de3fe-266e-4b1b-b3a6-ec948f8ee4fa tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:40.886986 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a53de3fe-266e-4b1b-b3a6-ec948f8ee4fa tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:40.887401 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a53de3fe-266e-4b1b-b3a6-ec948f8ee4fa tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:40.970226 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-a53de3fe-266e-4b1b-b3a6-ec948f8ee4fa tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lazy-loading 'services' on Instance uuid 6c129af4-e7f0-4743-85fd-5889547510f3 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:30:41.176714 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a53de3fe-266e-4b1b-b3a6-ec948f8ee4fa tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3/os-volume_attachments/a2a8915a-f50a-4a19-a782-4c5b781c61ab" status: 202 len: 0 microversion: 2.1 time: 0.302577 Sep 08 14:30:41.177278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1175/2631] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:30:40 2022] DELETE /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3/os-volume_attachments/a2a8915a-f50a-4a19-a782-4c5b781c61ab => generated 0 bytes in 304 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:30:41.699645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a620ec5c-551f-4ed4-a377-453d2e87ef82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:41.700241 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a620ec5c-551f-4ed4-a377-453d2e87ef82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:41.706770 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a620ec5c-551f-4ed4-a377-453d2e87ef82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:41.707251 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a620ec5c-551f-4ed4-a377-453d2e87ef82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:41.707689 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a620ec5c-551f-4ed4-a377-453d2e87ef82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:41.708374 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a620ec5c-551f-4ed4-a377-453d2e87ef82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:41.708865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a620ec5c-551f-4ed4-a377-453d2e87ef82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:41.709293 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a620ec5c-551f-4ed4-a377-453d2e87ef82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:41.848956 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-a620ec5c-551f-4ed4-a377-453d2e87ef82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:41.857052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a620ec5c-551f-4ed4-a377-453d2e87ef82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1361 microversion: 2.1 time: 0.161723 Sep 08 14:30:41.857954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1457/2632] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:41 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1361 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:42.473305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-57f7789d-0c62-4856-a5fc-c4bc28dadcdf req-c3d76974-9591-4f5a-a9e9-1b522f1513b2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c09b905a-0795-4dc5-a413-007f3951e49f", "name": "network-vif-plugged", "status": "completed", "tag": "c06eb27d-e611-488c-80be-0549fc42ecef"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:30:42.485673 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-57f7789d-0c62-4856-a5fc-c4bc28dadcdf req-c3d76974-9591-4f5a-a9e9-1b522f1513b2 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:42.486447 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-57f7789d-0c62-4856-a5fc-c4bc28dadcdf req-c3d76974-9591-4f5a-a9e9-1b522f1513b2 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:42.487052 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-57f7789d-0c62-4856-a5fc-c4bc28dadcdf req-c3d76974-9591-4f5a-a9e9-1b522f1513b2 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:42.500740 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-57f7789d-0c62-4856-a5fc-c4bc28dadcdf req-c3d76974-9591-4f5a-a9e9-1b522f1513b2 service nova] Creating event network-vif-plugged:c06eb27d-e611-488c-80be-0549fc42ecef for instance c09b905a-0795-4dc5-a413-007f3951e49f on hv-cinder-85646 Sep 08 14:30:42.505733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-57f7789d-0c62-4856-a5fc-c4bc28dadcdf req-c3d76974-9591-4f5a-a9e9-1b522f1513b2 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038717 Sep 08 14:30:42.506368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1176/2633] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:30:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:30:42.881968 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-38e41e98-5d33-4653-8042-a1eb2cdece4c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:42.882687 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-38e41e98-5d33-4653-8042-a1eb2cdece4c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:42.890042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38e41e98-5d33-4653-8042-a1eb2cdece4c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:42.890545 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38e41e98-5d33-4653-8042-a1eb2cdece4c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:42.891058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38e41e98-5d33-4653-8042-a1eb2cdece4c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:42.891674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38e41e98-5d33-4653-8042-a1eb2cdece4c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:42.892118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38e41e98-5d33-4653-8042-a1eb2cdece4c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:42.892543 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-38e41e98-5d33-4653-8042-a1eb2cdece4c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:43.022496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-38e41e98-5d33-4653-8042-a1eb2cdece4c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:43.030901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-38e41e98-5d33-4653-8042-a1eb2cdece4c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1361 microversion: 2.1 time: 0.153294 Sep 08 14:30:43.031498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1458/2634] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:42 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1361 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:43.375104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c6478bb0-7b10-47f7-9b32-9f4b85e4aba3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:43.375681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c6478bb0-7b10-47f7-9b32-9f4b85e4aba3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:43.377590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=47,nova_api:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:43.382477 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6478bb0-7b10-47f7-9b32-9f4b85e4aba3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:43.382951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6478bb0-7b10-47f7-9b32-9f4b85e4aba3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:43.383383 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6478bb0-7b10-47f7-9b32-9f4b85e4aba3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:43.384055 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6478bb0-7b10-47f7-9b32-9f4b85e4aba3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:43.384500 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6478bb0-7b10-47f7-9b32-9f4b85e4aba3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:43.384917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c6478bb0-7b10-47f7-9b32-9f4b85e4aba3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:43.411238 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c6478bb0-7b10-47f7-9b32-9f4b85e4aba3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:30:43.623440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c6478bb0-7b10-47f7-9b32-9f4b85e4aba3 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "DELETE /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 204 len: 0 microversion: 2.1 time: 0.252867 Sep 08 14:30:43.624173 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1177/2635] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:30:43 2022] DELETE /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:30:43.632355 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-88d31cad-4b7d-40c2-9fb2-397234b16f20 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:43.633008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-88d31cad-4b7d-40c2-9fb2-397234b16f20 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:43.639488 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-88d31cad-4b7d-40c2-9fb2-397234b16f20 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:43.640153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-88d31cad-4b7d-40c2-9fb2-397234b16f20 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:43.640834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-88d31cad-4b7d-40c2-9fb2-397234b16f20 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:43.641296 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-88d31cad-4b7d-40c2-9fb2-397234b16f20 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:43.641753 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-88d31cad-4b7d-40c2-9fb2-397234b16f20 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:43.642233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-88d31cad-4b7d-40c2-9fb2-397234b16f20 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:43.777795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-88d31cad-4b7d-40c2-9fb2-397234b16f20 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:43.786376 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-88d31cad-4b7d-40c2-9fb2-397234b16f20 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 200 len: 1512 microversion: 2.1 time: 0.157818 Sep 08 14:30:43.787326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1459/2636] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:43 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 1512 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:44.054084 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1760a599-89f1-46aa-b0a9-d67617fa52dd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:44.054706 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1760a599-89f1-46aa-b0a9-d67617fa52dd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:44.060619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1760a599-89f1-46aa-b0a9-d67617fa52dd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:44.061096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1760a599-89f1-46aa-b0a9-d67617fa52dd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:44.061523 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1760a599-89f1-46aa-b0a9-d67617fa52dd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:44.062225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1760a599-89f1-46aa-b0a9-d67617fa52dd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:44.062676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1760a599-89f1-46aa-b0a9-d67617fa52dd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:44.063096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1760a599-89f1-46aa-b0a9-d67617fa52dd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:44.192854 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1760a599-89f1-46aa-b0a9-d67617fa52dd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:44.201739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1760a599-89f1-46aa-b0a9-d67617fa52dd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1361 microversion: 2.1 time: 0.151928 Sep 08 14:30:44.202365 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1178/2637] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:44 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1361 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:44.809622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-59953fda-08be-46ee-bdf5-d31df24f2506 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:44.810269 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-59953fda-08be-46ee-bdf5-d31df24f2506 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:44.816681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-59953fda-08be-46ee-bdf5-d31df24f2506 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:44.817167 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-59953fda-08be-46ee-bdf5-d31df24f2506 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:44.817609 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-59953fda-08be-46ee-bdf5-d31df24f2506 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:44.818362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-59953fda-08be-46ee-bdf5-d31df24f2506 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:44.818825 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-59953fda-08be-46ee-bdf5-d31df24f2506 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:44.819254 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-59953fda-08be-46ee-bdf5-d31df24f2506 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:44.950993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-59953fda-08be-46ee-bdf5-d31df24f2506 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:44.958582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-59953fda-08be-46ee-bdf5-d31df24f2506 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 200 len: 1512 microversion: 2.1 time: 0.152824 Sep 08 14:30:44.959135 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1460/2638] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:44 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 1512 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:45.225153 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d2dca756-b0fe-4ae6-ae52-8932d40791c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:45.225700 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d2dca756-b0fe-4ae6-ae52-8932d40791c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:45.232260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d2dca756-b0fe-4ae6-ae52-8932d40791c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:45.232738 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d2dca756-b0fe-4ae6-ae52-8932d40791c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:45.233168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d2dca756-b0fe-4ae6-ae52-8932d40791c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:45.233844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d2dca756-b0fe-4ae6-ae52-8932d40791c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:45.234334 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d2dca756-b0fe-4ae6-ae52-8932d40791c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:45.234752 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d2dca756-b0fe-4ae6-ae52-8932d40791c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:45.365730 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d2dca756-b0fe-4ae6-ae52-8932d40791c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:45.373966 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d2dca756-b0fe-4ae6-ae52-8932d40791c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1361 microversion: 2.1 time: 0.152619 Sep 08 14:30:45.374569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1179/2639] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:45 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1361 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:45.982611 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e39da358-eef9-4a15-8814-ccd1b664b756 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:45.983578 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e39da358-eef9-4a15-8814-ccd1b664b756 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:45.990552 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e39da358-eef9-4a15-8814-ccd1b664b756 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:45.991045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e39da358-eef9-4a15-8814-ccd1b664b756 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:45.991486 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e39da358-eef9-4a15-8814-ccd1b664b756 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:45.992232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e39da358-eef9-4a15-8814-ccd1b664b756 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:45.992691 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e39da358-eef9-4a15-8814-ccd1b664b756 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:45.993121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e39da358-eef9-4a15-8814-ccd1b664b756 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:46.124377 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e39da358-eef9-4a15-8814-ccd1b664b756 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:46.132291 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e39da358-eef9-4a15-8814-ccd1b664b756 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 200 len: 1512 microversion: 2.1 time: 0.154171 Sep 08 14:30:46.132861 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1461/2640] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:45 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 1512 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:46.397995 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a32fc036-7ca9-4bfa-9a5a-c0f0d6200ae7 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:46.398648 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a32fc036-7ca9-4bfa-9a5a-c0f0d6200ae7 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:46.405697 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a32fc036-7ca9-4bfa-9a5a-c0f0d6200ae7 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:46.406411 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a32fc036-7ca9-4bfa-9a5a-c0f0d6200ae7 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:46.406854 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a32fc036-7ca9-4bfa-9a5a-c0f0d6200ae7 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:46.407553 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a32fc036-7ca9-4bfa-9a5a-c0f0d6200ae7 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:46.408014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a32fc036-7ca9-4bfa-9a5a-c0f0d6200ae7 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:46.408440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a32fc036-7ca9-4bfa-9a5a-c0f0d6200ae7 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:46.543706 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a32fc036-7ca9-4bfa-9a5a-c0f0d6200ae7 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:46.551338 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a32fc036-7ca9-4bfa-9a5a-c0f0d6200ae7 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1547 microversion: 2.1 time: 0.157426 Sep 08 14:30:46.551906 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1180/2641] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:46 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1547 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:46.574114 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-178ebfdf-92e9-478c-86f2-edaf737068b3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:46.574635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-178ebfdf-92e9-478c-86f2-edaf737068b3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:46.580894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-178ebfdf-92e9-478c-86f2-edaf737068b3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:46.581391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-178ebfdf-92e9-478c-86f2-edaf737068b3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:46.581832 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-178ebfdf-92e9-478c-86f2-edaf737068b3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:46.582562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-178ebfdf-92e9-478c-86f2-edaf737068b3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:46.583021 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-178ebfdf-92e9-478c-86f2-edaf737068b3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:46.583498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-178ebfdf-92e9-478c-86f2-edaf737068b3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:46.674519 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-178ebfdf-92e9-478c-86f2-edaf737068b3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.104743 Sep 08 14:30:46.675212 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1462/2642] 10.222.0.25 () {58 vars in 1275 bytes} [Thu Sep 8 14:30:46 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f/os-interface => generated 285 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:30:47.155484 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c68345b5-8486-4050-a6c3-79a6a1e3f653 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:47.156447 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c68345b5-8486-4050-a6c3-79a6a1e3f653 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:47.162938 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c68345b5-8486-4050-a6c3-79a6a1e3f653 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:47.163413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c68345b5-8486-4050-a6c3-79a6a1e3f653 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:47.163845 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c68345b5-8486-4050-a6c3-79a6a1e3f653 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:47.164530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c68345b5-8486-4050-a6c3-79a6a1e3f653 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:47.164977 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c68345b5-8486-4050-a6c3-79a6a1e3f653 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:47.165391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c68345b5-8486-4050-a6c3-79a6a1e3f653 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:47.306341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c68345b5-8486-4050-a6c3-79a6a1e3f653 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:47.315251 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c68345b5-8486-4050-a6c3-79a6a1e3f653 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 200 len: 1512 microversion: 2.1 time: 0.163836 Sep 08 14:30:47.323435 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1181/2643] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:47 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 1512 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:47.726025 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-5243e60b-bf75-495a-a632-6b724d1585b1 req-ea49da3d-266e-4f4b-8438-720bb5d4781b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6c129af4-e7f0-4743-85fd-5889547510f3", "name": "network-vif-deleted", "tag": "968b521a-c5c0-4679-87a8-ddd5842563b7"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:30:47.736878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-5243e60b-bf75-495a-a632-6b724d1585b1 req-ea49da3d-266e-4f4b-8438-720bb5d4781b service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:47.737454 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-5243e60b-bf75-495a-a632-6b724d1585b1 req-ea49da3d-266e-4f4b-8438-720bb5d4781b service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:47.737888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-5243e60b-bf75-495a-a632-6b724d1585b1 req-ea49da3d-266e-4f4b-8438-720bb5d4781b service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:47.750064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-5243e60b-bf75-495a-a632-6b724d1585b1 req-ea49da3d-266e-4f4b-8438-720bb5d4781b service nova] Creating event network-vif-deleted:968b521a-c5c0-4679-87a8-ddd5842563b7 for instance 6c129af4-e7f0-4743-85fd-5889547510f3 on hv-cinder-85646 Sep 08 14:30:47.755641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-5243e60b-bf75-495a-a632-6b724d1585b1 req-ea49da3d-266e-4f4b-8438-720bb5d4781b service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034001 Sep 08 14:30:47.756391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1463/2644] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:30:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:30:48.337621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e3f19961-e6ec-4473-a5e9-b4f21dd0af73 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:48.338385 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e3f19961-e6ec-4473-a5e9-b4f21dd0af73 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:48.346454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e3f19961-e6ec-4473-a5e9-b4f21dd0af73 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:48.347015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e3f19961-e6ec-4473-a5e9-b4f21dd0af73 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:48.347514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e3f19961-e6ec-4473-a5e9-b4f21dd0af73 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:48.348532 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e3f19961-e6ec-4473-a5e9-b4f21dd0af73 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:48.348938 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e3f19961-e6ec-4473-a5e9-b4f21dd0af73 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:48.349427 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e3f19961-e6ec-4473-a5e9-b4f21dd0af73 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:48.382668 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e3f19961-e6ec-4473-a5e9-b4f21dd0af73 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:48.383145 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e3f19961-e6ec-4473-a5e9-b4f21dd0af73 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:48.383578 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e3f19961-e6ec-4473-a5e9-b4f21dd0af73 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:48.384024 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-e3f19961-e6ec-4473-a5e9-b4f21dd0af73 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lazy-loading 'fault' on Instance uuid 6c129af4-e7f0-4743-85fd-5889547510f3 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:30:48.425519 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e3f19961-e6ec-4473-a5e9-b4f21dd0af73 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '5ae9377cf36e4086aa38c07b021fdead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1363660dde8f4e4295adc44c6ab87f8b', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:30:48.433928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e3f19961-e6ec-4473-a5e9-b4f21dd0af73 tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 200 len: 1309 microversion: 2.1 time: 0.100199 Sep 08 14:30:48.434524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1182/2645] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:48 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 1309 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:30:49.458486 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b7f73f84-dee5-45c7-84e3-a1c6cc6f91fd tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:49.459126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b7f73f84-dee5-45c7-84e3-a1c6cc6f91fd tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] [instance: 6c129af4-e7f0-4743-85fd-5889547510f3] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:49.465970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7f73f84-dee5-45c7-84e3-a1c6cc6f91fd tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:49.466478 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7f73f84-dee5-45c7-84e3-a1c6cc6f91fd tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:49.466916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7f73f84-dee5-45c7-84e3-a1c6cc6f91fd tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:49.467610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7f73f84-dee5-45c7-84e3-a1c6cc6f91fd tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:49.468064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7f73f84-dee5-45c7-84e3-a1c6cc6f91fd tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:49.468487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b7f73f84-dee5-45c7-84e3-a1c6cc6f91fd tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:49.481669 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-b7f73f84-dee5-45c7-84e3-a1c6cc6f91fd tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] HTTP exception thrown: Instance 6c129af4-e7f0-4743-85fd-5889547510f3 could not be found. Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b7f73f84-dee5-45c7-84e3-a1c6cc6f91fd tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Request method failure captured: Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 HTTP/1.1 Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfSNyJo8zI7plzmoT3joT1LB7boaKVSoM2Btn54mkWpdqceI6SLQFx4_Rpb5l-mqzqX661NrDxpQinv-CjzSmbNBVW650b02MoLz_tOe3RLJyylDgcU4AMsS5LTkC_9jOXC-JdkvqUTGjOl36FG-vYox8h_LoJX8tIcUOcoYc5HdH0s Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:30:49.483332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:30:49.485070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/1363660dde8f4e4295adc44c6ab87f8b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:30:49.485070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:30:49.485070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 1363660dde8f4e4295adc44c6ab87f8b Sep 08 14:30:49.485070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1362272129 Sep 08 14:30:49.485070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:30:49.485070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:30:49.485070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:30:49.485070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 5ae9377cf36e4086aa38c07b021fdead Sep 08 14:30:49.485070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-VolumesSnapshotTestJSON-1362272129-project Sep 08 14:30:49.485070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:30:49.485070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance 6c129af4-e7f0-4743-85fd-5889547510f3 could not be found. Sep 08 14:30:49.485070 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': '6c129af4-e7f0-4743-85fd-5889547510f3'} Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6c129af4-e7f0-4743-85fd-5889547510f3 could not be found. Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:30:49.486245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:30:49.488673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:30:49.488673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:30:49.488673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:30:49.488673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:30:49.488673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:30:49.488673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:30:49.488673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:30:49.488673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6c129af4-e7f0-4743-85fd-5889547510f3 could not be found. Sep 08 14:30:49.488673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:30:49.488673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:30:49.488673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:30:49.488673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:30:49.488673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:30:49.488673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:30:49.488673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:30:49.488673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:30:49.488673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6c129af4-e7f0-4743-85fd-5889547510f3 could not be found. Sep 08 14:30:49.488673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:30:49.490425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b7f73f84-dee5-45c7-84e3-a1c6cc6f91fd tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] Returning 404 to user: Instance 6c129af4-e7f0-4743-85fd-5889547510f3 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:30:49.490425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b7f73f84-dee5-45c7-84e3-a1c6cc6f91fd tempest-VolumesSnapshotTestJSON-1362272129 tempest-VolumesSnapshotTestJSON-1362272129-project] 10.222.0.25 "GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3" status: 404 len: 111 microversion: 2.1 time: 0.031400 Sep 08 14:30:49.490425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1464/2646] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:30:49 2022] GET /compute/v2.1/servers/6c129af4-e7f0-4743-85fd-5889547510f3 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:30:49.786317 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-56024b60-cda0-4002-b5ba-98645f02a473 req-4365be59-11cc-4eee-aeaa-66c6224b1af4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c09b905a-0795-4dc5-a413-007f3951e49f", "tag": "c06eb27d-e611-488c-80be-0549fc42ecef"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:30:49.796676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-56024b60-cda0-4002-b5ba-98645f02a473 req-4365be59-11cc-4eee-aeaa-66c6224b1af4 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:49.797148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-56024b60-cda0-4002-b5ba-98645f02a473 req-4365be59-11cc-4eee-aeaa-66c6224b1af4 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:49.797570 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-56024b60-cda0-4002-b5ba-98645f02a473 req-4365be59-11cc-4eee-aeaa-66c6224b1af4 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:49.809139 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-56024b60-cda0-4002-b5ba-98645f02a473 req-4365be59-11cc-4eee-aeaa-66c6224b1af4 service nova] Creating event network-changed:c06eb27d-e611-488c-80be-0549fc42ecef for instance c09b905a-0795-4dc5-a413-007f3951e49f on hv-cinder-85646 Sep 08 14:30:49.814317 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-56024b60-cda0-4002-b5ba-98645f02a473 req-4365be59-11cc-4eee-aeaa-66c6224b1af4 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031900 Sep 08 14:30:49.814902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1183/2647] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:30:49 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:30:57.894716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f2c27db5-b0b1-4fe2-9492-f74e2b702c5a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:57.895301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f2c27db5-b0b1-4fe2-9492-f74e2b702c5a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:57.897290 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=45,nova_api:INSERT=3 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:30:57.901979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2c27db5-b0b1-4fe2-9492-f74e2b702c5a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:57.902493 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2c27db5-b0b1-4fe2-9492-f74e2b702c5a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:57.902928 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2c27db5-b0b1-4fe2-9492-f74e2b702c5a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:57.903634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2c27db5-b0b1-4fe2-9492-f74e2b702c5a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:57.904081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2c27db5-b0b1-4fe2-9492-f74e2b702c5a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:57.905253 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f2c27db5-b0b1-4fe2-9492-f74e2b702c5a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:57.928330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f2c27db5-b0b1-4fe2-9492-f74e2b702c5a tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4/ips" status: 200 len: 150 microversion: 2.1 time: 0.037519 Sep 08 14:30:57.928923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1465/2648] 10.222.0.25 () {58 vars in 1248 bytes} [Thu Sep 8 14:30:57 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4/ips => generated 150 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:30:59.561049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-181db6c2-2ede-4cb4-9294-3dab3a08468a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:30:59.562130 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-181db6c2-2ede-4cb4-9294-3dab3a08468a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:59.568602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-181db6c2-2ede-4cb4-9294-3dab3a08468a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:59.569083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-181db6c2-2ede-4cb4-9294-3dab3a08468a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:59.569509 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-181db6c2-2ede-4cb4-9294-3dab3a08468a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:59.570236 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-181db6c2-2ede-4cb4-9294-3dab3a08468a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:59.570680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-181db6c2-2ede-4cb4-9294-3dab3a08468a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:59.571093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-181db6c2-2ede-4cb4-9294-3dab3a08468a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:59.594593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-181db6c2-2ede-4cb4-9294-3dab3a08468a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9/ips" status: 200 len: 149 microversion: 2.20 time: 0.038771 Sep 08 14:30:59.595179 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1184/2649] 10.222.0.25 () {60 vars in 1289 bytes} [Thu Sep 8 14:30:59 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9/ips => generated 149 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:30:59.760772 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-61200b34-ab28-412a-b4a3-59d537db5de1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "6c913abd-dc50-45a5-af8b-e8dfaaf34293"}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:30:59.763099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-61200b34-ab28-412a-b4a3-59d537db5de1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:30:59.770775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61200b34-ab28-412a-b4a3-59d537db5de1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:59.771260 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61200b34-ab28-412a-b4a3-59d537db5de1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:59.771684 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61200b34-ab28-412a-b4a3-59d537db5de1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:59.772364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61200b34-ab28-412a-b4a3-59d537db5de1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:30:59.772810 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61200b34-ab28-412a-b4a3-59d537db5de1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:30:59.773229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61200b34-ab28-412a-b4a3-59d537db5de1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:30:59.776783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=82 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:00.690350 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-61200b34-ab28-412a-b4a3-59d537db5de1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "POST /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.933597 Sep 08 14:31:00.691042 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1466/2650] 10.222.0.25 () {60 vars in 1323 bytes} [Thu Sep 8 14:30:59 2022] POST /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4/os-volume_attachments => generated 194 bytes in 935 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:31:02.377203 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a5793a5c-e414-4bc3-93dd-f9e2acf5cd1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:31:02.377780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a5793a5c-e414-4bc3-93dd-f9e2acf5cd1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:02.384125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5793a5c-e414-4bc3-93dd-f9e2acf5cd1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:02.384599 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5793a5c-e414-4bc3-93dd-f9e2acf5cd1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:02.385021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5793a5c-e414-4bc3-93dd-f9e2acf5cd1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:02.385687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5793a5c-e414-4bc3-93dd-f9e2acf5cd1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:02.386130 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5793a5c-e414-4bc3-93dd-f9e2acf5cd1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:02.386572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5793a5c-e414-4bc3-93dd-f9e2acf5cd1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:02.409997 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-a5793a5c-e414-4bc3-93dd-f9e2acf5cd1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lazy-loading 'flavor' on Instance uuid b0741665-d24b-4c17-93c7-90bcfff633b9 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:31:02.823004 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a5793a5c-e414-4bc3-93dd-f9e2acf5cd1c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "POST /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9/action" status: 202 len: 0 microversion: 2.20 time: 0.449741 Sep 08 14:31:02.823619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1185/2651] 10.222.0.25 () {62 vars in 1319 bytes} [Thu Sep 8 14:31:02 2022] POST /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9/action => generated 0 bytes in 451 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:31:02.836193 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-b6b4bbaf-e257-4e2a-afbb-ab656914c3ac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:02.837656 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-b6b4bbaf-e257-4e2a-afbb-ab656914c3ac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:02.845940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b6b4bbaf-e257-4e2a-afbb-ab656914c3ac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:02.846608 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b6b4bbaf-e257-4e2a-afbb-ab656914c3ac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:02.854104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b6b4bbaf-e257-4e2a-afbb-ab656914c3ac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:02.854104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b6b4bbaf-e257-4e2a-afbb-ab656914c3ac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:02.854104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b6b4bbaf-e257-4e2a-afbb-ab656914c3ac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:02.854104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-b6b4bbaf-e257-4e2a-afbb-ab656914c3ac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:02.987411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b6b4bbaf-e257-4e2a-afbb-ab656914c3ac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:02.995549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b6b4bbaf-e257-4e2a-afbb-ab656914c3ac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:02.996983 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-b6b4bbaf-e257-4e2a-afbb-ab656914c3ac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:02.998183 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-b6b4bbaf-e257-4e2a-afbb-ab656914c3ac tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.167308 Sep 08 14:31:02.998711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1467/2652] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:02 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:04.025295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8612a8f8-81d2-4fda-804a-1998200e2167 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:04.025988 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8612a8f8-81d2-4fda-804a-1998200e2167 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:04.032614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8612a8f8-81d2-4fda-804a-1998200e2167 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:04.033093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8612a8f8-81d2-4fda-804a-1998200e2167 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:04.033522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8612a8f8-81d2-4fda-804a-1998200e2167 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:04.034292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8612a8f8-81d2-4fda-804a-1998200e2167 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:04.034816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8612a8f8-81d2-4fda-804a-1998200e2167 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:04.035239 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8612a8f8-81d2-4fda-804a-1998200e2167 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:04.172650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8612a8f8-81d2-4fda-804a-1998200e2167 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:04.177377 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=3,nova_cell1:UPDATE=3 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:04.182675 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8612a8f8-81d2-4fda-804a-1998200e2167 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:04.184117 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8612a8f8-81d2-4fda-804a-1998200e2167 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:04.185281 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8612a8f8-81d2-4fda-804a-1998200e2167 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.164823 Sep 08 14:31:04.185834 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1186/2653] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:04 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:05.211293 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4ea101ea-88db-429f-9193-9fa417fc7453 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:05.212001 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4ea101ea-88db-429f-9193-9fa417fc7453 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:05.218650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ea101ea-88db-429f-9193-9fa417fc7453 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:05.223764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ea101ea-88db-429f-9193-9fa417fc7453 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:05.223764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ea101ea-88db-429f-9193-9fa417fc7453 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:05.223764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ea101ea-88db-429f-9193-9fa417fc7453 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:05.223764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ea101ea-88db-429f-9193-9fa417fc7453 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:05.224208 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4ea101ea-88db-429f-9193-9fa417fc7453 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:05.453656 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4ea101ea-88db-429f-9193-9fa417fc7453 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:05.463032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4ea101ea-88db-429f-9193-9fa417fc7453 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:05.464483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4ea101ea-88db-429f-9193-9fa417fc7453 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:05.465869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4ea101ea-88db-429f-9193-9fa417fc7453 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.258544 Sep 08 14:31:05.466860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1468/2654] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:05 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:06.492666 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9af93753-e861-4935-8232-9dda28135ae0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:06.493274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9af93753-e861-4935-8232-9dda28135ae0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:06.499649 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9af93753-e861-4935-8232-9dda28135ae0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:06.500128 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9af93753-e861-4935-8232-9dda28135ae0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:06.500553 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9af93753-e861-4935-8232-9dda28135ae0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:06.501368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9af93753-e861-4935-8232-9dda28135ae0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:06.501815 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9af93753-e861-4935-8232-9dda28135ae0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:06.502272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9af93753-e861-4935-8232-9dda28135ae0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:06.639384 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9af93753-e861-4935-8232-9dda28135ae0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:06.648377 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9af93753-e861-4935-8232-9dda28135ae0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:06.649822 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9af93753-e861-4935-8232-9dda28135ae0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:06.651020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9af93753-e861-4935-8232-9dda28135ae0 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.163072 Sep 08 14:31:06.651613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1187/2655] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:06 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:07.675796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6c7b0b93-f198-4ce3-8792-66a4efae59de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:07.676347 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6c7b0b93-f198-4ce3-8792-66a4efae59de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:07.682732 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c7b0b93-f198-4ce3-8792-66a4efae59de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:07.683211 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c7b0b93-f198-4ce3-8792-66a4efae59de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:07.683644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c7b0b93-f198-4ce3-8792-66a4efae59de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:07.684335 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c7b0b93-f198-4ce3-8792-66a4efae59de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:07.684783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c7b0b93-f198-4ce3-8792-66a4efae59de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:07.685201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c7b0b93-f198-4ce3-8792-66a4efae59de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:07.821137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6c7b0b93-f198-4ce3-8792-66a4efae59de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:07.835580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6c7b0b93-f198-4ce3-8792-66a4efae59de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:07.838476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6c7b0b93-f198-4ce3-8792-66a4efae59de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:07.840158 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6c7b0b93-f198-4ce3-8792-66a4efae59de tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.168290 Sep 08 14:31:07.840998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1469/2656] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:07 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:08.866307 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-13416a1e-10f7-4fe4-9dd7-fd265a1bc944 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:08.867005 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-13416a1e-10f7-4fe4-9dd7-fd265a1bc944 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:08.873691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-13416a1e-10f7-4fe4-9dd7-fd265a1bc944 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:08.874202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-13416a1e-10f7-4fe4-9dd7-fd265a1bc944 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:08.874636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-13416a1e-10f7-4fe4-9dd7-fd265a1bc944 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:08.875326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-13416a1e-10f7-4fe4-9dd7-fd265a1bc944 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:08.875778 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-13416a1e-10f7-4fe4-9dd7-fd265a1bc944 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:08.876197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-13416a1e-10f7-4fe4-9dd7-fd265a1bc944 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:09.008834 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-13416a1e-10f7-4fe4-9dd7-fd265a1bc944 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:09.017582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-13416a1e-10f7-4fe4-9dd7-fd265a1bc944 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:09.019051 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-13416a1e-10f7-4fe4-9dd7-fd265a1bc944 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:09.020260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-13416a1e-10f7-4fe4-9dd7-fd265a1bc944 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.158010 Sep 08 14:31:09.020817 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1188/2657] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:08 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:10.045305 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-22e9e685-9575-4c42-b62d-7c63e8d2c258 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:10.045977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-22e9e685-9575-4c42-b62d-7c63e8d2c258 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:10.053116 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22e9e685-9575-4c42-b62d-7c63e8d2c258 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:10.053592 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22e9e685-9575-4c42-b62d-7c63e8d2c258 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:10.054024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22e9e685-9575-4c42-b62d-7c63e8d2c258 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:10.054730 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22e9e685-9575-4c42-b62d-7c63e8d2c258 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:10.055178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22e9e685-9575-4c42-b62d-7c63e8d2c258 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:10.055598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22e9e685-9575-4c42-b62d-7c63e8d2c258 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:10.184299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-22e9e685-9575-4c42-b62d-7c63e8d2c258 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:10.193226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-22e9e685-9575-4c42-b62d-7c63e8d2c258 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:10.194736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-22e9e685-9575-4c42-b62d-7c63e8d2c258 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:10.195904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-22e9e685-9575-4c42-b62d-7c63e8d2c258 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.154664 Sep 08 14:31:10.196466 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1470/2658] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:10 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:11.221348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-984d90eb-3c18-4138-9c21-7c530bab8dd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:11.221914 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-984d90eb-3c18-4138-9c21-7c530bab8dd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:11.228603 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-984d90eb-3c18-4138-9c21-7c530bab8dd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:11.229093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-984d90eb-3c18-4138-9c21-7c530bab8dd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:11.229524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-984d90eb-3c18-4138-9c21-7c530bab8dd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:11.230229 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-984d90eb-3c18-4138-9c21-7c530bab8dd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:11.230681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-984d90eb-3c18-4138-9c21-7c530bab8dd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:11.231104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-984d90eb-3c18-4138-9c21-7c530bab8dd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:11.365582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-984d90eb-3c18-4138-9c21-7c530bab8dd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:11.373790 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-984d90eb-3c18-4138-9c21-7c530bab8dd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:11.375250 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-984d90eb-3c18-4138-9c21-7c530bab8dd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:11.376950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-984d90eb-3c18-4138-9c21-7c530bab8dd3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.159813 Sep 08 14:31:11.377498 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1189/2659] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:11 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:12.403280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-84d0d6b7-81c7-4469-b643-6f62af1886a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:12.403912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-84d0d6b7-81c7-4469-b643-6f62af1886a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:12.411072 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84d0d6b7-81c7-4469-b643-6f62af1886a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:12.411564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84d0d6b7-81c7-4469-b643-6f62af1886a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:12.412008 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84d0d6b7-81c7-4469-b643-6f62af1886a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:12.412713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84d0d6b7-81c7-4469-b643-6f62af1886a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:12.413173 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84d0d6b7-81c7-4469-b643-6f62af1886a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:12.413605 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84d0d6b7-81c7-4469-b643-6f62af1886a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:12.548401 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-84d0d6b7-81c7-4469-b643-6f62af1886a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:12.556419 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-84d0d6b7-81c7-4469-b643-6f62af1886a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:12.557916 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-84d0d6b7-81c7-4469-b643-6f62af1886a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:12.559105 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-84d0d6b7-81c7-4469-b643-6f62af1886a7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.159867 Sep 08 14:31:12.559658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1471/2660] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:12 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:12.998095 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8bf5943a-94b2-4393-8d5f-1cb882a24685 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:12.998625 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8bf5943a-94b2-4393-8d5f-1cb882a24685 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:13.005092 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8bf5943a-94b2-4393-8d5f-1cb882a24685 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:13.009293 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8bf5943a-94b2-4393-8d5f-1cb882a24685 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:13.009293 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8bf5943a-94b2-4393-8d5f-1cb882a24685 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:13.009293 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8bf5943a-94b2-4393-8d5f-1cb882a24685 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:13.009293 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8bf5943a-94b2-4393-8d5f-1cb882a24685 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:13.009782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8bf5943a-94b2-4393-8d5f-1cb882a24685 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:13.095683 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-8bf5943a-94b2-4393-8d5f-1cb882a24685 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lazy-loading 'services' on Instance uuid bcef9a84-43d0-440b-804b-4a32893fc0d4 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:31:13.360471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8bf5943a-94b2-4393-8d5f-1cb882a24685 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4/os-volume_attachments/6c913abd-dc50-45a5-af8b-e8dfaaf34293" status: 202 len: 0 microversion: 2.1 time: 0.366103 Sep 08 14:31:13.361112 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1190/2661] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:31:12 2022] DELETE /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4/os-volume_attachments/6c913abd-dc50-45a5-af8b-e8dfaaf34293 => generated 0 bytes in 368 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 08 14:31:13.369821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-127ad156-d461-49e8-809f-7f17a6d6866f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:13.372435 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-127ad156-d461-49e8-809f-7f17a6d6866f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:13.379205 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-127ad156-d461-49e8-809f-7f17a6d6866f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:13.379700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-127ad156-d461-49e8-809f-7f17a6d6866f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:13.382954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-127ad156-d461-49e8-809f-7f17a6d6866f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:13.382954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-127ad156-d461-49e8-809f-7f17a6d6866f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:13.382954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-127ad156-d461-49e8-809f-7f17a6d6866f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:13.382954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-127ad156-d461-49e8-809f-7f17a6d6866f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:13.418634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-127ad156-d461-49e8-809f-7f17a6d6866f tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052935 Sep 08 14:31:13.419335 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1472/2662] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:31:13 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:31:13.585891 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e401d804-e835-4254-b159-e15bcb793c98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:13.586619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e401d804-e835-4254-b159-e15bcb793c98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:13.588554 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=35,nova_api:UPDATE=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:13.593607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e401d804-e835-4254-b159-e15bcb793c98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:13.594091 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e401d804-e835-4254-b159-e15bcb793c98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:13.600512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e401d804-e835-4254-b159-e15bcb793c98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:13.600512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e401d804-e835-4254-b159-e15bcb793c98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:13.600512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e401d804-e835-4254-b159-e15bcb793c98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:13.600512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e401d804-e835-4254-b159-e15bcb793c98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:13.730783 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e401d804-e835-4254-b159-e15bcb793c98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:13.739433 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e401d804-e835-4254-b159-e15bcb793c98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:13.740879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e401d804-e835-4254-b159-e15bcb793c98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:13.742036 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e401d804-e835-4254-b159-e15bcb793c98 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.160438 Sep 08 14:31:13.742632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1191/2663] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:13 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:14.434923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1f010b97-2687-4a00-9b7d-b455bb0771b5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:14.437141 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1f010b97-2687-4a00-9b7d-b455bb0771b5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:14.444573 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1f010b97-2687-4a00-9b7d-b455bb0771b5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:14.450583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1f010b97-2687-4a00-9b7d-b455bb0771b5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:14.450583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1f010b97-2687-4a00-9b7d-b455bb0771b5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:14.450583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1f010b97-2687-4a00-9b7d-b455bb0771b5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:14.450583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1f010b97-2687-4a00-9b7d-b455bb0771b5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:14.451032 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1f010b97-2687-4a00-9b7d-b455bb0771b5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:14.479625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1f010b97-2687-4a00-9b7d-b455bb0771b5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048999 Sep 08 14:31:14.490739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1473/2664] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:31:14 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:31:14.768428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5f55a9f6-1180-4c6c-b0fd-975326ff586d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:14.768979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5f55a9f6-1180-4c6c-b0fd-975326ff586d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:14.775317 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5f55a9f6-1180-4c6c-b0fd-975326ff586d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:14.775791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5f55a9f6-1180-4c6c-b0fd-975326ff586d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:14.776291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5f55a9f6-1180-4c6c-b0fd-975326ff586d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:14.776887 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5f55a9f6-1180-4c6c-b0fd-975326ff586d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:14.777326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5f55a9f6-1180-4c6c-b0fd-975326ff586d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:14.777739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5f55a9f6-1180-4c6c-b0fd-975326ff586d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:14.913706 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5f55a9f6-1180-4c6c-b0fd-975326ff586d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:14.922579 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5f55a9f6-1180-4c6c-b0fd-975326ff586d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:14.924125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5f55a9f6-1180-4c6c-b0fd-975326ff586d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:14.925810 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5f55a9f6-1180-4c6c-b0fd-975326ff586d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.161373 Sep 08 14:31:14.926404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1192/2665] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:14 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:15.492458 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-57041b2b-2084-477f-8b4a-0165385af0ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:15.494590 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-57041b2b-2084-477f-8b4a-0165385af0ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:15.501087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57041b2b-2084-477f-8b4a-0165385af0ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:15.501582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57041b2b-2084-477f-8b4a-0165385af0ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:15.502027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57041b2b-2084-477f-8b4a-0165385af0ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:15.502809 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57041b2b-2084-477f-8b4a-0165385af0ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:15.503271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57041b2b-2084-477f-8b4a-0165385af0ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:15.503699 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-57041b2b-2084-477f-8b4a-0165385af0ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:15.535627 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-57041b2b-2084-477f-8b4a-0165385af0ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047270 Sep 08 14:31:15.536185 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1474/2666] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:31:15 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 08 14:31:15.605649 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6d853fdf-6b3b-492d-bfed-8e1abb5db7f1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:15.606147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6d853fdf-6b3b-492d-bfed-8e1abb5db7f1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:15.613385 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d853fdf-6b3b-492d-bfed-8e1abb5db7f1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:15.614447 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d853fdf-6b3b-492d-bfed-8e1abb5db7f1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:15.614447 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d853fdf-6b3b-492d-bfed-8e1abb5db7f1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:15.615165 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d853fdf-6b3b-492d-bfed-8e1abb5db7f1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:15.615652 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d853fdf-6b3b-492d-bfed-8e1abb5db7f1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:15.616093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d853fdf-6b3b-492d-bfed-8e1abb5db7f1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:15.644152 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6d853fdf-6b3b-492d-bfed-8e1abb5db7f1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:31:15.951132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-32af8161-26a5-45db-b9bb-25285adb59c7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:15.951734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-32af8161-26a5-45db-b9bb-25285adb59c7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:15.958547 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32af8161-26a5-45db-b9bb-25285adb59c7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:15.959024 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32af8161-26a5-45db-b9bb-25285adb59c7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:15.959459 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32af8161-26a5-45db-b9bb-25285adb59c7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:15.960149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32af8161-26a5-45db-b9bb-25285adb59c7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:15.960613 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32af8161-26a5-45db-b9bb-25285adb59c7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:15.960847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6d853fdf-6b3b-492d-bfed-8e1abb5db7f1 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 204 len: 0 microversion: 2.1 time: 0.358862 Sep 08 14:31:15.961317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32af8161-26a5-45db-b9bb-25285adb59c7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:15.961758 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1193/2667] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:31:15 2022] DELETE /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:31:15.970827 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-64c7e397-5180-49db-8130-eac74ade686b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:15.972202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-64c7e397-5180-49db-8130-eac74ade686b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:15.978379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64c7e397-5180-49db-8130-eac74ade686b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:15.978862 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64c7e397-5180-49db-8130-eac74ade686b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:15.979294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64c7e397-5180-49db-8130-eac74ade686b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:15.980050 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64c7e397-5180-49db-8130-eac74ade686b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:15.980496 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64c7e397-5180-49db-8130-eac74ade686b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:15.980990 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64c7e397-5180-49db-8130-eac74ade686b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:16.115082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-32af8161-26a5-45db-b9bb-25285adb59c7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:16.127591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-32af8161-26a5-45db-b9bb-25285adb59c7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:16.129047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-32af8161-26a5-45db-b9bb-25285adb59c7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:16.130340 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-32af8161-26a5-45db-b9bb-25285adb59c7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.183185 Sep 08 14:31:16.130917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1475/2668] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:15 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 14:31:16.134393 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-64c7e397-5180-49db-8130-eac74ade686b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:16.143847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-64c7e397-5180-49db-8130-eac74ade686b tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1681 microversion: 2.1 time: 0.177528 Sep 08 14:31:16.150530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1194/2669] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:15 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1681 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:31:17.157297 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-770af303-2ea2-4479-b51e-6bc360b7fe85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:17.157933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-770af303-2ea2-4479-b51e-6bc360b7fe85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:17.165536 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-770af303-2ea2-4479-b51e-6bc360b7fe85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:17.166020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-770af303-2ea2-4479-b51e-6bc360b7fe85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:17.166615 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-770af303-2ea2-4479-b51e-6bc360b7fe85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:17.167323 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-770af303-2ea2-4479-b51e-6bc360b7fe85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:17.167871 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-770af303-2ea2-4479-b51e-6bc360b7fe85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:17.168342 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-770af303-2ea2-4479-b51e-6bc360b7fe85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:17.172912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f70a50e3-4c7a-4c4a-b141-efd2ec32017d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:17.177104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f70a50e3-4c7a-4c4a-b141-efd2ec32017d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:17.180410 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f70a50e3-4c7a-4c4a-b141-efd2ec32017d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:17.180918 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f70a50e3-4c7a-4c4a-b141-efd2ec32017d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:17.181342 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f70a50e3-4c7a-4c4a-b141-efd2ec32017d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:17.182023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f70a50e3-4c7a-4c4a-b141-efd2ec32017d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:17.182495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f70a50e3-4c7a-4c4a-b141-efd2ec32017d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:17.182912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f70a50e3-4c7a-4c4a-b141-efd2ec32017d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:17.328989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f70a50e3-4c7a-4c4a-b141-efd2ec32017d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:17.331923 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-770af303-2ea2-4479-b51e-6bc360b7fe85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:17.337537 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f70a50e3-4c7a-4c4a-b141-efd2ec32017d tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1681 microversion: 2.1 time: 0.168420 Sep 08 14:31:17.343909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1195/2670] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:17 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1681 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:31:17.344064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-770af303-2ea2-4479-b51e-6bc360b7fe85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:17.344064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-770af303-2ea2-4479-b51e-6bc360b7fe85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:17.344064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-770af303-2ea2-4479-b51e-6bc360b7fe85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.190186 Sep 08 14:31:17.344398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1476/2671] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:17 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:18.362538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7782b514-3d96-4f04-a3b5-370b96af22ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:18.363113 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7782b514-3d96-4f04-a3b5-370b96af22ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:18.368991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f1927f37-75fb-4244-842c-12be9f4225af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:18.369538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f1927f37-75fb-4244-842c-12be9f4225af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:18.369699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7782b514-3d96-4f04-a3b5-370b96af22ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:18.370328 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7782b514-3d96-4f04-a3b5-370b96af22ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:18.370808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7782b514-3d96-4f04-a3b5-370b96af22ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:18.371570 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7782b514-3d96-4f04-a3b5-370b96af22ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:18.372017 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7782b514-3d96-4f04-a3b5-370b96af22ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:18.372432 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7782b514-3d96-4f04-a3b5-370b96af22ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:18.376591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f1927f37-75fb-4244-842c-12be9f4225af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:18.377089 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f1927f37-75fb-4244-842c-12be9f4225af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:18.378275 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f1927f37-75fb-4244-842c-12be9f4225af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:18.378986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f1927f37-75fb-4244-842c-12be9f4225af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:18.379450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f1927f37-75fb-4244-842c-12be9f4225af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:18.379886 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f1927f37-75fb-4244-842c-12be9f4225af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:18.514397 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7782b514-3d96-4f04-a3b5-370b96af22ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:18.520546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f1927f37-75fb-4244-842c-12be9f4225af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:18.522540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7782b514-3d96-4f04-a3b5-370b96af22ee tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1681 microversion: 2.1 time: 0.164849 Sep 08 14:31:18.523126 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1196/2672] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:18 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1681 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:31:18.528986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f1927f37-75fb-4244-842c-12be9f4225af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:18.531376 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f1927f37-75fb-4244-842c-12be9f4225af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:18.531652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f1927f37-75fb-4244-842c-12be9f4225af tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.166624 Sep 08 14:31:18.532207 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1477/2673] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:18 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:19.547063 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-59c6a9ad-f77e-4599-a137-03f1f031c967 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:19.547690 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-59c6a9ad-f77e-4599-a137-03f1f031c967 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:19.553820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59c6a9ad-f77e-4599-a137-03f1f031c967 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:19.554364 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59c6a9ad-f77e-4599-a137-03f1f031c967 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:19.554807 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59c6a9ad-f77e-4599-a137-03f1f031c967 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:19.555568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59c6a9ad-f77e-4599-a137-03f1f031c967 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:19.556382 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59c6a9ad-f77e-4599-a137-03f1f031c967 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:19.556817 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-59c6a9ad-f77e-4599-a137-03f1f031c967 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:19.557482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ea32f694-73c7-49b6-8b0e-75ae7a0422bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:19.558094 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ea32f694-73c7-49b6-8b0e-75ae7a0422bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:19.565242 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ea32f694-73c7-49b6-8b0e-75ae7a0422bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:19.565796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ea32f694-73c7-49b6-8b0e-75ae7a0422bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:19.566283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ea32f694-73c7-49b6-8b0e-75ae7a0422bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:19.566991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ea32f694-73c7-49b6-8b0e-75ae7a0422bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:19.567446 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ea32f694-73c7-49b6-8b0e-75ae7a0422bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:19.567873 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ea32f694-73c7-49b6-8b0e-75ae7a0422bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:19.701495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-59c6a9ad-f77e-4599-a137-03f1f031c967 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:19.709600 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-59c6a9ad-f77e-4599-a137-03f1f031c967 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1681 microversion: 2.1 time: 0.167148 Sep 08 14:31:19.710457 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1197/2674] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:19 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1681 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:31:19.870979 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ea32f694-73c7-49b6-8b0e-75ae7a0422bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:19.880867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ea32f694-73c7-49b6-8b0e-75ae7a0422bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:19.882607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ea32f694-73c7-49b6-8b0e-75ae7a0422bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:19.883896 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ea32f694-73c7-49b6-8b0e-75ae7a0422bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.330278 Sep 08 14:31:19.884455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1478/2675] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:19 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 14:31:20.106956 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-3a76defb-b904-49b5-bcf0-c1e6eafb8186 req-d7b08a96-f0e9-4f61-a7cc-0583298fdd61 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bcef9a84-43d0-440b-804b-4a32893fc0d4", "name": "network-vif-deleted", "tag": "c960c128-6e5a-4241-bfed-484d84bfbc84"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:31:20.121196 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-3a76defb-b904-49b5-bcf0-c1e6eafb8186 req-d7b08a96-f0e9-4f61-a7cc-0583298fdd61 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:20.132368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-3a76defb-b904-49b5-bcf0-c1e6eafb8186 req-d7b08a96-f0e9-4f61-a7cc-0583298fdd61 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:20.132368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-3a76defb-b904-49b5-bcf0-c1e6eafb8186 req-d7b08a96-f0e9-4f61-a7cc-0583298fdd61 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:20.144529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-3a76defb-b904-49b5-bcf0-c1e6eafb8186 req-d7b08a96-f0e9-4f61-a7cc-0583298fdd61 service nova] Creating event network-vif-deleted:c960c128-6e5a-4241-bfed-484d84bfbc84 for instance bcef9a84-43d0-440b-804b-4a32893fc0d4 on hv-cinder-85646 Sep 08 14:31:20.149716 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-3a76defb-b904-49b5-bcf0-c1e6eafb8186 req-d7b08a96-f0e9-4f61-a7cc-0583298fdd61 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.047822 Sep 08 14:31:20.150689 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1198/2676] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:31:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:31:20.733305 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e6818b4a-936f-4b59-96f2-9bcc60d581b3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:20.734154 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e6818b4a-936f-4b59-96f2-9bcc60d581b3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:20.741645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6818b4a-936f-4b59-96f2-9bcc60d581b3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:20.742132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6818b4a-936f-4b59-96f2-9bcc60d581b3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:20.742603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6818b4a-936f-4b59-96f2-9bcc60d581b3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:20.743331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6818b4a-936f-4b59-96f2-9bcc60d581b3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:20.743871 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6818b4a-936f-4b59-96f2-9bcc60d581b3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:20.744240 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6818b4a-936f-4b59-96f2-9bcc60d581b3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:20.781416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6818b4a-936f-4b59-96f2-9bcc60d581b3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:20.781913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6818b4a-936f-4b59-96f2-9bcc60d581b3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:20.782398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e6818b4a-936f-4b59-96f2-9bcc60d581b3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:20.782862 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-e6818b4a-936f-4b59-96f2-9bcc60d581b3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lazy-loading 'fault' on Instance uuid bcef9a84-43d0-440b-804b-4a32893fc0d4 {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:31:20.831476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e6818b4a-936f-4b59-96f2-9bcc60d581b3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'eca89f435299415c8448931e73f5b0fc', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a4e0fba6cab34c429c73fcfac7ebfd94', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:20.840339 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e6818b4a-936f-4b59-96f2-9bcc60d581b3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 200 len: 1331 microversion: 2.1 time: 0.111135 Sep 08 14:31:20.842938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1479/2677] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:20 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 1331 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:31:20.926602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4d3121ae-3d44-4963-89d2-2dcbf3f48f7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:20.926602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4d3121ae-3d44-4963-89d2-2dcbf3f48f7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:20.926602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d3121ae-3d44-4963-89d2-2dcbf3f48f7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:20.926602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d3121ae-3d44-4963-89d2-2dcbf3f48f7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:20.927332 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d3121ae-3d44-4963-89d2-2dcbf3f48f7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:20.927332 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d3121ae-3d44-4963-89d2-2dcbf3f48f7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:20.927332 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d3121ae-3d44-4963-89d2-2dcbf3f48f7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:20.927332 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d3121ae-3d44-4963-89d2-2dcbf3f48f7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:21.093536 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4d3121ae-3d44-4963-89d2-2dcbf3f48f7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:21.104077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4d3121ae-3d44-4963-89d2-2dcbf3f48f7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:21.105909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4d3121ae-3d44-4963-89d2-2dcbf3f48f7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:21.107415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4d3121ae-3d44-4963-89d2-2dcbf3f48f7b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.198232 Sep 08 14:31:21.118316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1199/2678] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:20 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 14:31:21.864288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2eeccf47-cbcf-4fd8-9109-3443b755a5e0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:21.864900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2eeccf47-cbcf-4fd8-9109-3443b755a5e0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:21.871604 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2eeccf47-cbcf-4fd8-9109-3443b755a5e0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:21.872095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2eeccf47-cbcf-4fd8-9109-3443b755a5e0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:21.872534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2eeccf47-cbcf-4fd8-9109-3443b755a5e0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:21.873238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2eeccf47-cbcf-4fd8-9109-3443b755a5e0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:21.873696 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2eeccf47-cbcf-4fd8-9109-3443b755a5e0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:21.874126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2eeccf47-cbcf-4fd8-9109-3443b755a5e0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:21.887467 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-2eeccf47-cbcf-4fd8-9109-3443b755a5e0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance bcef9a84-43d0-440b-804b-4a32893fc0d4 could not be found. Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2eeccf47-cbcf-4fd8-9109-3443b755a5e0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 HTTP/1.1 Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:31:21.889394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:31:21.891411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:31:21.891411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:21.891411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:31:21.891411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:21.891411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:31:21.891411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:31:21.891411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:31:21.891411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:31:21.891411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:31:21.891411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:31:21.891411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance bcef9a84-43d0-440b-804b-4a32893fc0d4 could not be found. Sep 08 14:31:21.891411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'bcef9a84-43d0-440b-804b-4a32893fc0d4'} Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bcef9a84-43d0-440b-804b-4a32893fc0d4 could not be found. Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:31:21.892630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:31:21.894416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:31:21.894416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:31:21.894416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:31:21.894416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:31:21.894416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:31:21.894416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:31:21.894416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:31:21.894416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bcef9a84-43d0-440b-804b-4a32893fc0d4 could not be found. Sep 08 14:31:21.894416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:21.894416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:31:21.894416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:21.894416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:21.894416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:31:21.894416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:31:21.894416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:31:21.894416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:31:21.894416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bcef9a84-43d0-440b-804b-4a32893fc0d4 could not be found. Sep 08 14:31:21.894416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:21.895961 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2eeccf47-cbcf-4fd8-9109-3443b755a5e0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance bcef9a84-43d0-440b-804b-4a32893fc0d4 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:31:21.895961 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2eeccf47-cbcf-4fd8-9109-3443b755a5e0 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 404 len: 111 microversion: 2.1 time: 0.031073 Sep 08 14:31:21.895961 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1480/2679] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:21 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 08 14:31:21.900429 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3bd372c6-d314-44f3-9402-5353bda683f5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '.version_select at 0x7fce0220d820>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:21.996696 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3bd372c6-d314-44f3-9402-5353bda683f5 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1145077434" status: 202 len: 0 microversion: 2.1 time: 0.100022 Sep 08 14:31:21.997398 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1200/2680] 10.222.0.25 () {58 vars in 1221 bytes} [Thu Sep 8 14:31:21 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1145077434 => generated 0 bytes in 101 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:31:22.135374 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ed1d4cc0-5643-449e-9213-f6e2cfc2a9e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:22.135991 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ed1d4cc0-5643-449e-9213-f6e2cfc2a9e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:22.142674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed1d4cc0-5643-449e-9213-f6e2cfc2a9e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:22.143624 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed1d4cc0-5643-449e-9213-f6e2cfc2a9e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:22.144065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed1d4cc0-5643-449e-9213-f6e2cfc2a9e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:22.144762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed1d4cc0-5643-449e-9213-f6e2cfc2a9e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:22.145216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed1d4cc0-5643-449e-9213-f6e2cfc2a9e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:22.145663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ed1d4cc0-5643-449e-9213-f6e2cfc2a9e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:22.319901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ed1d4cc0-5643-449e-9213-f6e2cfc2a9e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:22.322266 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:22.329087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ed1d4cc0-5643-449e-9213-f6e2cfc2a9e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:22.331264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ed1d4cc0-5643-449e-9213-f6e2cfc2a9e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:22.332505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ed1d4cc0-5643-449e-9213-f6e2cfc2a9e7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.201062 Sep 08 14:31:22.333081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1481/2681] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:22 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:23.358420 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-963f95c0-1f71-49b8-8a17-1dcdcc39ecb8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:23.359041 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-963f95c0-1f71-49b8-8a17-1dcdcc39ecb8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:23.365799 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-963f95c0-1f71-49b8-8a17-1dcdcc39ecb8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:23.366308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-963f95c0-1f71-49b8-8a17-1dcdcc39ecb8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:23.366734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-963f95c0-1f71-49b8-8a17-1dcdcc39ecb8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:23.367445 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-963f95c0-1f71-49b8-8a17-1dcdcc39ecb8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:23.367958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-963f95c0-1f71-49b8-8a17-1dcdcc39ecb8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:23.368380 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-963f95c0-1f71-49b8-8a17-1dcdcc39ecb8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:23.498476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-963f95c0-1f71-49b8-8a17-1dcdcc39ecb8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:23.507274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-963f95c0-1f71-49b8-8a17-1dcdcc39ecb8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:23.508719 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-963f95c0-1f71-49b8-8a17-1dcdcc39ecb8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:23.509875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-963f95c0-1f71-49b8-8a17-1dcdcc39ecb8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.156021 Sep 08 14:31:23.510471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1201/2682] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:23 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:24.050924 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-706d2d90-04d1-440f-8ff3-eaf3cdf44618 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:24.051441 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-706d2d90-04d1-440f-8ff3-eaf3cdf44618 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:24.058596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-706d2d90-04d1-440f-8ff3-eaf3cdf44618 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:24.059086 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-706d2d90-04d1-440f-8ff3-eaf3cdf44618 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:24.059530 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-706d2d90-04d1-440f-8ff3-eaf3cdf44618 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:24.060250 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-706d2d90-04d1-440f-8ff3-eaf3cdf44618 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:24.060716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-706d2d90-04d1-440f-8ff3-eaf3cdf44618 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:24.061145 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-706d2d90-04d1-440f-8ff3-eaf3cdf44618 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:24.074606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-706d2d90-04d1-440f-8ff3-eaf3cdf44618 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance bcef9a84-43d0-440b-804b-4a32893fc0d4 could not be found. Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-706d2d90-04d1-440f-8ff3-eaf3cdf44618 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 HTTP/1.1 Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:31:24.076156 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:31:24.077997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:31:24.077997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:24.077997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:31:24.077997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:24.077997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:31:24.077997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:31:24.077997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:31:24.077997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:31:24.077997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:31:24.077997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:31:24.077997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance bcef9a84-43d0-440b-804b-4a32893fc0d4 could not be found. Sep 08 14:31:24.077997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'bcef9a84-43d0-440b-804b-4a32893fc0d4'} Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bcef9a84-43d0-440b-804b-4a32893fc0d4 could not be found. Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:31:24.078987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bcef9a84-43d0-440b-804b-4a32893fc0d4 could not be found. Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bcef9a84-43d0-440b-804b-4a32893fc0d4 could not be found. Sep 08 14:31:24.080685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:24.082634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-706d2d90-04d1-440f-8ff3-eaf3cdf44618 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance bcef9a84-43d0-440b-804b-4a32893fc0d4 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:31:24.082634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-706d2d90-04d1-440f-8ff3-eaf3cdf44618 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 404 len: 111 microversion: 2.1 time: 0.031544 Sep 08 14:31:24.082634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1482/2683] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:31:24 2022] DELETE /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:31:24.087265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b2b79f04-f707-4dc0-a5f8-388dab439cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:24.087902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b2b79f04-f707-4dc0-a5f8-388dab439cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: bcef9a84-43d0-440b-804b-4a32893fc0d4] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:24.093895 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b2b79f04-f707-4dc0-a5f8-388dab439cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:24.094413 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b2b79f04-f707-4dc0-a5f8-388dab439cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:24.094846 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b2b79f04-f707-4dc0-a5f8-388dab439cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:24.095525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b2b79f04-f707-4dc0-a5f8-388dab439cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:24.096490 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b2b79f04-f707-4dc0-a5f8-388dab439cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:24.097062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b2b79f04-f707-4dc0-a5f8-388dab439cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:24.110298 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-b2b79f04-f707-4dc0-a5f8-388dab439cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance bcef9a84-43d0-440b-804b-4a32893fc0d4 could not be found. Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b2b79f04-f707-4dc0-a5f8-388dab439cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 HTTP/1.1 Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:31:24.115471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:31:24.117222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:31:24.117222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:24.117222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:31:24.117222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:24.117222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:31:24.117222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:31:24.117222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:31:24.117222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:31:24.117222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:31:24.117222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:31:24.117222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance bcef9a84-43d0-440b-804b-4a32893fc0d4 could not be found. Sep 08 14:31:24.117222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'bcef9a84-43d0-440b-804b-4a32893fc0d4'} Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bcef9a84-43d0-440b-804b-4a32893fc0d4 could not be found. Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:31:24.118404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:31:24.119823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:31:24.119823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:31:24.119823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:31:24.119823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:31:24.119823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:31:24.119823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:31:24.119823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:31:24.119823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bcef9a84-43d0-440b-804b-4a32893fc0d4 could not be found. Sep 08 14:31:24.119823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:24.119823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:31:24.119823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:24.119823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:24.119823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:31:24.119823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:31:24.119823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:31:24.119823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:31:24.119823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bcef9a84-43d0-440b-804b-4a32893fc0d4 could not be found. Sep 08 14:31:24.119823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:24.121732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b2b79f04-f707-4dc0-a5f8-388dab439cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance bcef9a84-43d0-440b-804b-4a32893fc0d4 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:31:24.121732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b2b79f04-f707-4dc0-a5f8-388dab439cdf tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4" status: 404 len: 111 microversion: 2.1 time: 0.030579 Sep 08 14:31:24.121732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1202/2684] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:24 2022] GET /compute/v2.1/servers/bcef9a84-43d0-440b-804b-4a32893fc0d4 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:31:24.537869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-675889dc-44ca-4d83-b392-85534422b36b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:24.538411 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-675889dc-44ca-4d83-b392-85534422b36b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:24.545037 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-675889dc-44ca-4d83-b392-85534422b36b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:24.545519 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-675889dc-44ca-4d83-b392-85534422b36b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:24.545949 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-675889dc-44ca-4d83-b392-85534422b36b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:24.546664 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-675889dc-44ca-4d83-b392-85534422b36b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:24.547113 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-675889dc-44ca-4d83-b392-85534422b36b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:24.547564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-675889dc-44ca-4d83-b392-85534422b36b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:24.676303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-675889dc-44ca-4d83-b392-85534422b36b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:24.684787 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-675889dc-44ca-4d83-b392-85534422b36b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:24.686242 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-675889dc-44ca-4d83-b392-85534422b36b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:24.687386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-675889dc-44ca-4d83-b392-85534422b36b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.154131 Sep 08 14:31:24.687939 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1483/2685] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:24 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 14:31:25.363925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-240906ec-e361-4429-b5a5-48e5882e90e4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:25.364429 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-240906ec-e361-4429-b5a5-48e5882e90e4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:25.370805 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-240906ec-e361-4429-b5a5-48e5882e90e4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:25.371279 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-240906ec-e361-4429-b5a5-48e5882e90e4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:25.371712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-240906ec-e361-4429-b5a5-48e5882e90e4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:25.372389 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-240906ec-e361-4429-b5a5-48e5882e90e4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:25.372837 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-240906ec-e361-4429-b5a5-48e5882e90e4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:25.373260 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-240906ec-e361-4429-b5a5-48e5882e90e4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:25.386622 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-240906ec-e361-4429-b5a5-48e5882e90e4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-240906ec-e361-4429-b5a5-48e5882e90e4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 HTTP/1.1 Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:31:25.388183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:31:25.390513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:31:25.390513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:25.390513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:31:25.390513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:25.390513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:31:25.390513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:31:25.390513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:31:25.390513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:31:25.390513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:31:25.390513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:31:25.390513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:31:25.390513 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '5a9f0e69-51d0-4a26-89d2-91c47d197556'} Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:31:25.391655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:31:25.393282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.394969 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-240906ec-e361-4429-b5a5-48e5882e90e4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:31:25.394969 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-240906ec-e361-4429-b5a5-48e5882e90e4 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 404 len: 111 microversion: 2.1 time: 0.030570 Sep 08 14:31:25.394969 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1203/2686] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:31:25 2022] DELETE /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:31:25.400112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ca983498-07a5-44a5-98c7-619c5f650837 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:25.400722 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ca983498-07a5-44a5-98c7-619c5f650837 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 5a9f0e69-51d0-4a26-89d2-91c47d197556] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:25.407197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca983498-07a5-44a5-98c7-619c5f650837 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:25.407674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca983498-07a5-44a5-98c7-619c5f650837 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:25.408106 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca983498-07a5-44a5-98c7-619c5f650837 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:25.409359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca983498-07a5-44a5-98c7-619c5f650837 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:25.409802 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca983498-07a5-44a5-98c7-619c5f650837 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:25.410254 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ca983498-07a5-44a5-98c7-619c5f650837 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:25.425929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-ca983498-07a5-44a5-98c7-619c5f650837 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ca983498-07a5-44a5-98c7-619c5f650837 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 HTTP/1.1 Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:31:25.427498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:31:25.429324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:31:25.429324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:25.429324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:31:25.429324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:25.429324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:31:25.429324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:31:25.429324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:31:25.429324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:31:25.429324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:31:25.429324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:31:25.429324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:31:25.429324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': '5a9f0e69-51d0-4a26-89d2-91c47d197556'} Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:31:25.430637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:31:25.432563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:31:25.432563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:31:25.432563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:31:25.432563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:31:25.432563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:31:25.432563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:31:25.432563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:31:25.432563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:31:25.432563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.432563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:31:25.432563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.432563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:25.432563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:31:25.432563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:31:25.432563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:31:25.432563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:31:25.432563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. Sep 08 14:31:25.432563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.434321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ca983498-07a5-44a5-98c7-619c5f650837 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance 5a9f0e69-51d0-4a26-89d2-91c47d197556 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:31:25.434321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ca983498-07a5-44a5-98c7-619c5f650837 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556" status: 404 len: 111 microversion: 2.1 time: 0.032534 Sep 08 14:31:25.434321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1484/2687] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:25 2022] GET /compute/v2.1/servers/5a9f0e69-51d0-4a26-89d2-91c47d197556 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:31:25.437290 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d51d6ec2-9588-4ef1-8e92-a0a9765fa014 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:25.437812 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d51d6ec2-9588-4ef1-8e92-a0a9765fa014 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:25.443873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d51d6ec2-9588-4ef1-8e92-a0a9765fa014 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:25.444382 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d51d6ec2-9588-4ef1-8e92-a0a9765fa014 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:25.444820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d51d6ec2-9588-4ef1-8e92-a0a9765fa014 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:25.445503 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d51d6ec2-9588-4ef1-8e92-a0a9765fa014 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:25.445951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d51d6ec2-9588-4ef1-8e92-a0a9765fa014 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:25.446401 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d51d6ec2-9588-4ef1-8e92-a0a9765fa014 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:25.449158 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:25.460638 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-d51d6ec2-9588-4ef1-8e92-a0a9765fa014 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 could not be found. Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d51d6ec2-9588-4ef1-8e92-a0a9765fa014 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 HTTP/1.1 Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:31:25.462200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:31:25.464067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:31:25.464067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:25.464067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:31:25.464067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:25.464067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:31:25.464067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:31:25.464067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:31:25.464067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:31:25.464067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:31:25.464067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:31:25.464067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 could not be found. Sep 08 14:31:25.464067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '82890cea-b89a-4020-b9aa-8a7d94ac6005'} Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 could not be found. Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:31:25.465154 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 could not be found. Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 could not be found. Sep 08 14:31:25.466735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.468203 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d51d6ec2-9588-4ef1-8e92-a0a9765fa014 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:31:25.468203 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d51d6ec2-9588-4ef1-8e92-a0a9765fa014 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 404 len: 111 microversion: 2.1 time: 0.030226 Sep 08 14:31:25.468203 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1204/2688] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:31:25 2022] DELETE /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:31:25.472729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bcef6c7c-2112-4a8e-a0b6-a6aca6aa4154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:25.473337 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bcef6c7c-2112-4a8e-a0b6-a6aca6aa4154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 82890cea-b89a-4020-b9aa-8a7d94ac6005] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:25.479249 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcef6c7c-2112-4a8e-a0b6-a6aca6aa4154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:25.479721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcef6c7c-2112-4a8e-a0b6-a6aca6aa4154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:25.480149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcef6c7c-2112-4a8e-a0b6-a6aca6aa4154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:25.480824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcef6c7c-2112-4a8e-a0b6-a6aca6aa4154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:25.481271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcef6c7c-2112-4a8e-a0b6-a6aca6aa4154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:25.481691 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcef6c7c-2112-4a8e-a0b6-a6aca6aa4154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:25.495829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-bcef6c7c-2112-4a8e-a0b6-a6aca6aa4154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 could not be found. Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bcef6c7c-2112-4a8e-a0b6-a6aca6aa4154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 HTTP/1.1 Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:31:25.497318 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:31:25.499150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:31:25.499150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:25.499150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:31:25.499150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:25.499150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:31:25.499150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:31:25.499150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:31:25.499150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:31:25.499150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:31:25.499150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:31:25.499150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 could not be found. Sep 08 14:31:25.499150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': '82890cea-b89a-4020-b9aa-8a7d94ac6005'} Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 could not be found. Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:31:25.500058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:31:25.501487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:31:25.501487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:31:25.501487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:31:25.501487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:31:25.501487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:31:25.501487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:31:25.501487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:31:25.501487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 could not be found. Sep 08 14:31:25.501487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.501487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:31:25.501487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.501487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:25.501487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:31:25.501487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:31:25.501487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:31:25.501487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:31:25.501487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 could not be found. Sep 08 14:31:25.501487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:25.502889 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bcef6c7c-2112-4a8e-a0b6-a6aca6aa4154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance 82890cea-b89a-4020-b9aa-8a7d94ac6005 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:31:25.502889 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bcef6c7c-2112-4a8e-a0b6-a6aca6aa4154 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005" status: 404 len: 111 microversion: 2.1 time: 0.030438 Sep 08 14:31:25.502889 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1485/2689] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:25 2022] GET /compute/v2.1/servers/82890cea-b89a-4020-b9aa-8a7d94ac6005 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 08 14:31:25.712934 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1413b1b9-73a2-4430-8adf-537761488c9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:25.713536 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1413b1b9-73a2-4430-8adf-537761488c9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:25.720671 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1413b1b9-73a2-4430-8adf-537761488c9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:25.721796 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1413b1b9-73a2-4430-8adf-537761488c9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:25.722254 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1413b1b9-73a2-4430-8adf-537761488c9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:25.722945 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1413b1b9-73a2-4430-8adf-537761488c9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:25.723392 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1413b1b9-73a2-4430-8adf-537761488c9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:25.723840 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1413b1b9-73a2-4430-8adf-537761488c9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:25.861923 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1413b1b9-73a2-4430-8adf-537761488c9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:25.870021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1413b1b9-73a2-4430-8adf-537761488c9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:25.871472 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1413b1b9-73a2-4430-8adf-537761488c9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:25.872710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1413b1b9-73a2-4430-8adf-537761488c9d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1740 microversion: 2.20 time: 0.163716 Sep 08 14:31:25.873259 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1205/2690] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:25 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1740 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:26.804863 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-91279f25-b065-40f6-a010-dd133de47e25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:26.805428 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-91279f25-b065-40f6-a010-dd133de47e25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:26.812434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91279f25-b065-40f6-a010-dd133de47e25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:26.813007 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91279f25-b065-40f6-a010-dd133de47e25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:26.813513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91279f25-b065-40f6-a010-dd133de47e25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:26.814326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91279f25-b065-40f6-a010-dd133de47e25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:26.815437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91279f25-b065-40f6-a010-dd133de47e25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:26.815910 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91279f25-b065-40f6-a010-dd133de47e25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:26.830056 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-91279f25-b065-40f6-a010-dd133de47e25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 could not be found. Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-91279f25-b065-40f6-a010-dd133de47e25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 HTTP/1.1 Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:31:26.831633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:31:26.833300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:31:26.833300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:26.833300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:31:26.833300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:26.833300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:31:26.833300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:31:26.833300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:31:26.833300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:31:26.833300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:31:26.833300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:31:26.833300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 could not be found. Sep 08 14:31:26.833300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': '483c7eb7-3827-403f-84f3-c7c65dcbe574'} Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 could not be found. Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:31:26.834308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 could not be found. Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 could not be found. Sep 08 14:31:26.835793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:31:26.837289 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-91279f25-b065-40f6-a010-dd133de47e25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:31:26.837289 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-91279f25-b065-40f6-a010-dd133de47e25 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "DELETE /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 404 len: 111 microversion: 2.1 time: 0.032310 Sep 08 14:31:26.837289 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1486/2691] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:31:26 2022] DELETE /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:31:26.846954 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1bbf4bb2-1e4c-4062-a881-f6d9b18869a3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:26.847682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1bbf4bb2-1e4c-4062-a881-f6d9b18869a3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] [instance: 483c7eb7-3827-403f-84f3-c7c65dcbe574] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:26.854871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1bbf4bb2-1e4c-4062-a881-f6d9b18869a3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:26.855414 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1bbf4bb2-1e4c-4062-a881-f6d9b18869a3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:26.855976 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1bbf4bb2-1e4c-4062-a881-f6d9b18869a3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:26.856705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1bbf4bb2-1e4c-4062-a881-f6d9b18869a3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:26.857177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1bbf4bb2-1e4c-4062-a881-f6d9b18869a3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:26.857593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1bbf4bb2-1e4c-4062-a881-f6d9b18869a3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:26.873498 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-1bbf4bb2-1e4c-4062-a881-f6d9b18869a3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] HTTP exception thrown: Instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 could not be found. Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1bbf4bb2-1e4c-4062-a881-f6d9b18869a3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Request method failure captured: Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 HTTP/1.1 Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPnJwdLJx9AmHmFf8p8r1FHQD_dOs-uzxnD5S_01LyT8ZejQJTTaO8Tv8Omi9HGdtRRinAT9dT0QL95ZjFSnU8UtATF2ThyaVpsz3Qf8CjAlDtgSDA8kIFjoemje0aRES7fn0xisIBmGcVMZVgL_W6bUiNdmbUtcN1pE0rKrLs5Je8 Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:31:26.875094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:31:26.876851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/a4e0fba6cab34c429c73fcfac7ebfd94"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:31:26.876851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:26.876851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: a4e0fba6cab34c429c73fcfac7ebfd94 Sep 08 14:31:26.876851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-35295268 Sep 08 14:31:26.876851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:31:26.876851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:31:26.876851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:31:26.876851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: eca89f435299415c8448931e73f5b0fc Sep 08 14:31:26.876851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeNegativeTest-35295268-project Sep 08 14:31:26.876851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:31:26.876851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 could not be found. Sep 08 14:31:26.876851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '483c7eb7-3827-403f-84f3-c7c65dcbe574'} Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 could not be found. Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:31:26.877888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:31:26.879562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:31:26.879562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:31:26.879562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:31:26.879562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:31:26.879562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:31:26.879562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:31:26.879562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:31:26.879562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 could not be found. Sep 08 14:31:26.879562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:26.879562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:31:26.879562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:26.879562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:31:26.879562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:31:26.879562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:31:26.879562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:31:26.879562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:31:26.879562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 could not be found. Sep 08 14:31:26.879562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:31:26.881060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1bbf4bb2-1e4c-4062-a881-f6d9b18869a3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] Returning 404 to user: Instance 483c7eb7-3827-403f-84f3-c7c65dcbe574 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:31:26.881060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1bbf4bb2-1e4c-4062-a881-f6d9b18869a3 tempest-AttachVolumeNegativeTest-35295268 tempest-AttachVolumeNegativeTest-35295268-project] 10.222.0.25 "GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574" status: 404 len: 111 microversion: 2.1 time: 0.034081 Sep 08 14:31:26.881060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1206/2692] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:26 2022] GET /compute/v2.1/servers/483c7eb7-3827-403f-84f3-c7c65dcbe574 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:31:26.898295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6276a5de-3ad0-4874-8f96-e931b8cbeef8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:26.899020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6276a5de-3ad0-4874-8f96-e931b8cbeef8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:26.905398 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6276a5de-3ad0-4874-8f96-e931b8cbeef8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:26.905726 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6276a5de-3ad0-4874-8f96-e931b8cbeef8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:26.906274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6276a5de-3ad0-4874-8f96-e931b8cbeef8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:26.907054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6276a5de-3ad0-4874-8f96-e931b8cbeef8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:26.907594 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6276a5de-3ad0-4874-8f96-e931b8cbeef8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:26.908089 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6276a5de-3ad0-4874-8f96-e931b8cbeef8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:27.045159 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6276a5de-3ad0-4874-8f96-e931b8cbeef8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:27.053743 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6276a5de-3ad0-4874-8f96-e931b8cbeef8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:27.055212 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6276a5de-3ad0-4874-8f96-e931b8cbeef8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:27.056380 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6276a5de-3ad0-4874-8f96-e931b8cbeef8 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1740 microversion: 2.20 time: 0.162091 Sep 08 14:31:27.056931 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1487/2693] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:26 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1740 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:28.081213 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-793bd4e0-02f8-4518-b83d-09fd39b05b8f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:28.081967 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-793bd4e0-02f8-4518-b83d-09fd39b05b8f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:28.090772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-793bd4e0-02f8-4518-b83d-09fd39b05b8f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:28.091271 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-793bd4e0-02f8-4518-b83d-09fd39b05b8f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:28.091721 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-793bd4e0-02f8-4518-b83d-09fd39b05b8f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:28.092448 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-793bd4e0-02f8-4518-b83d-09fd39b05b8f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:28.092902 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-793bd4e0-02f8-4518-b83d-09fd39b05b8f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:28.095403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-793bd4e0-02f8-4518-b83d-09fd39b05b8f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:28.297921 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-793bd4e0-02f8-4518-b83d-09fd39b05b8f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:28.307218 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-793bd4e0-02f8-4518-b83d-09fd39b05b8f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:28.309087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-793bd4e0-02f8-4518-b83d-09fd39b05b8f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:28.311156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-793bd4e0-02f8-4518-b83d-09fd39b05b8f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1735 microversion: 2.20 time: 0.233774 Sep 08 14:31:28.315217 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1207/2694] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:28 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1735 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:28.469286 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-7128ec7d-54c2-4848-9f91-8ccd0c9753c3 req-cfeee2ac-b516-4ce4-8bbf-8c81dfec89de service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b0741665-d24b-4c17-93c7-90bcfff633b9", "name": "network-vif-plugged", "status": "completed", "tag": "67fe80b7-8ae7-46be-8809-bd4958623c19"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:31:28.481775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-7128ec7d-54c2-4848-9f91-8ccd0c9753c3 req-cfeee2ac-b516-4ce4-8bbf-8c81dfec89de service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:28.482432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-7128ec7d-54c2-4848-9f91-8ccd0c9753c3 req-cfeee2ac-b516-4ce4-8bbf-8c81dfec89de service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:28.482967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-7128ec7d-54c2-4848-9f91-8ccd0c9753c3 req-cfeee2ac-b516-4ce4-8bbf-8c81dfec89de service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:28.497188 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-7128ec7d-54c2-4848-9f91-8ccd0c9753c3 req-cfeee2ac-b516-4ce4-8bbf-8c81dfec89de service nova] Creating event network-vif-plugged:67fe80b7-8ae7-46be-8809-bd4958623c19 for instance b0741665-d24b-4c17-93c7-90bcfff633b9 on hv-cinder-85646 Sep 08 14:31:28.503303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-7128ec7d-54c2-4848-9f91-8ccd0c9753c3 req-cfeee2ac-b516-4ce4-8bbf-8c81dfec89de service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036976 Sep 08 14:31:28.504450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1488/2695] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:31:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:31:29.352050 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-26959dcf-3ad2-437b-a616-a20524f2483d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:29.353429 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-26959dcf-3ad2-437b-a616-a20524f2483d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:29.364773 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26959dcf-3ad2-437b-a616-a20524f2483d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:29.366371 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26959dcf-3ad2-437b-a616-a20524f2483d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:29.371338 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26959dcf-3ad2-437b-a616-a20524f2483d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:29.372343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26959dcf-3ad2-437b-a616-a20524f2483d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:29.373049 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26959dcf-3ad2-437b-a616-a20524f2483d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:29.373715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-26959dcf-3ad2-437b-a616-a20524f2483d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:29.602535 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-26959dcf-3ad2-437b-a616-a20524f2483d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:29.612833 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-26959dcf-3ad2-437b-a616-a20524f2483d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:29.614640 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-26959dcf-3ad2-437b-a616-a20524f2483d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:29.616072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-26959dcf-3ad2-437b-a616-a20524f2483d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1735 microversion: 2.20 time: 0.268031 Sep 08 14:31:29.617131 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1208/2696] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:29 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1735 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:30.644130 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-be6f2b3a-3643-40e3-a2ae-3d83f648be4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:30.644837 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-be6f2b3a-3643-40e3-a2ae-3d83f648be4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:30.646849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=50 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:30.657273 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be6f2b3a-3643-40e3-a2ae-3d83f648be4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:30.661700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be6f2b3a-3643-40e3-a2ae-3d83f648be4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:30.661700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be6f2b3a-3643-40e3-a2ae-3d83f648be4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:30.661700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be6f2b3a-3643-40e3-a2ae-3d83f648be4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:30.662644 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be6f2b3a-3643-40e3-a2ae-3d83f648be4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:30.663412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-be6f2b3a-3643-40e3-a2ae-3d83f648be4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:30.972332 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-be6f2b3a-3643-40e3-a2ae-3d83f648be4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:30.988909 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-be6f2b3a-3643-40e3-a2ae-3d83f648be4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:30.990536 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-be6f2b3a-3643-40e3-a2ae-3d83f648be4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:30.991881 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-be6f2b3a-3643-40e3-a2ae-3d83f648be4c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1735 microversion: 2.20 time: 0.353620 Sep 08 14:31:30.997827 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1489/2697] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:30 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1735 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:32.023087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b1302041-04f1-4016-8b6a-4a4e9faada28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:32.023629 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b1302041-04f1-4016-8b6a-4a4e9faada28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:32.030272 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1302041-04f1-4016-8b6a-4a4e9faada28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:32.030771 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1302041-04f1-4016-8b6a-4a4e9faada28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:32.043187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1302041-04f1-4016-8b6a-4a4e9faada28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:32.043187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1302041-04f1-4016-8b6a-4a4e9faada28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:32.043187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1302041-04f1-4016-8b6a-4a4e9faada28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:32.043187 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b1302041-04f1-4016-8b6a-4a4e9faada28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:32.186984 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b1302041-04f1-4016-8b6a-4a4e9faada28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:32.196192 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b1302041-04f1-4016-8b6a-4a4e9faada28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:32.197634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b1302041-04f1-4016-8b6a-4a4e9faada28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:32.198851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b1302041-04f1-4016-8b6a-4a4e9faada28 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1735 microversion: 2.20 time: 0.179994 Sep 08 14:31:32.199440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1209/2698] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:32 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1735 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:33.224704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7008fcb5-92c8-49fb-8917-4fbf7d37bf00 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:33.225451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7008fcb5-92c8-49fb-8917-4fbf7d37bf00 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:33.232856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7008fcb5-92c8-49fb-8917-4fbf7d37bf00 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:33.233343 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7008fcb5-92c8-49fb-8917-4fbf7d37bf00 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:33.243704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7008fcb5-92c8-49fb-8917-4fbf7d37bf00 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:33.244952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7008fcb5-92c8-49fb-8917-4fbf7d37bf00 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:33.251224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7008fcb5-92c8-49fb-8917-4fbf7d37bf00 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:33.251224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7008fcb5-92c8-49fb-8917-4fbf7d37bf00 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:33.440860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7008fcb5-92c8-49fb-8917-4fbf7d37bf00 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:33.452740 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7008fcb5-92c8-49fb-8917-4fbf7d37bf00 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:33.454601 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7008fcb5-92c8-49fb-8917-4fbf7d37bf00 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:33.456004 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7008fcb5-92c8-49fb-8917-4fbf7d37bf00 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1735 microversion: 2.20 time: 0.235768 Sep 08 14:31:33.457279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1490/2699] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:33 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1735 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:34.490020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7bcdc4dc-3e03-4baf-88b4-2ab604b962f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:34.490020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7bcdc4dc-3e03-4baf-88b4-2ab604b962f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:34.497411 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7bcdc4dc-3e03-4baf-88b4-2ab604b962f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:34.498115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7bcdc4dc-3e03-4baf-88b4-2ab604b962f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:34.499087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7bcdc4dc-3e03-4baf-88b4-2ab604b962f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:34.501031 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7bcdc4dc-3e03-4baf-88b4-2ab604b962f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:34.501997 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7bcdc4dc-3e03-4baf-88b4-2ab604b962f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:34.503196 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7bcdc4dc-3e03-4baf-88b4-2ab604b962f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:34.677769 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7bcdc4dc-3e03-4baf-88b4-2ab604b962f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:34.687454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7bcdc4dc-3e03-4baf-88b4-2ab604b962f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:34.689626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7bcdc4dc-3e03-4baf-88b4-2ab604b962f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:34.692102 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7bcdc4dc-3e03-4baf-88b4-2ab604b962f1 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1735 microversion: 2.20 time: 0.209021 Sep 08 14:31:34.693087 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1210/2700] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:34 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1735 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:34.909372 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8dd7e629-0f80-4d72-a552-f67eca876693 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:34.910552 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8dd7e629-0f80-4d72-a552-f67eca876693 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:34.917638 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8dd7e629-0f80-4d72-a552-f67eca876693 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:34.918452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8dd7e629-0f80-4d72-a552-f67eca876693 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:34.919103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8dd7e629-0f80-4d72-a552-f67eca876693 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:34.920018 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8dd7e629-0f80-4d72-a552-f67eca876693 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:34.920650 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8dd7e629-0f80-4d72-a552-f67eca876693 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:34.921266 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8dd7e629-0f80-4d72-a552-f67eca876693 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:34.953777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8dd7e629-0f80-4d72-a552-f67eca876693 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f/ips" status: 200 len: 146 microversion: 2.1 time: 0.048282 Sep 08 14:31:34.961857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1491/2701] 10.222.0.25 () {58 vars in 1248 bytes} [Thu Sep 8 14:31:34 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f/ips => generated 146 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:31:35.718811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-302f5378-4bbe-4a80-b052-44cfcf271357 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:35.719422 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-302f5378-4bbe-4a80-b052-44cfcf271357 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:35.726698 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-302f5378-4bbe-4a80-b052-44cfcf271357 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:35.727168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-302f5378-4bbe-4a80-b052-44cfcf271357 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:35.727602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-302f5378-4bbe-4a80-b052-44cfcf271357 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:35.728290 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-302f5378-4bbe-4a80-b052-44cfcf271357 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:35.728741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-302f5378-4bbe-4a80-b052-44cfcf271357 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:35.729261 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-302f5378-4bbe-4a80-b052-44cfcf271357 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:35.868202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-302f5378-4bbe-4a80-b052-44cfcf271357 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:35.881179 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-302f5378-4bbe-4a80-b052-44cfcf271357 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:35.882789 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-302f5378-4bbe-4a80-b052-44cfcf271357 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:35.884109 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-302f5378-4bbe-4a80-b052-44cfcf271357 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1735 microversion: 2.20 time: 0.169246 Sep 08 14:31:35.902436 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1211/2702] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:35 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1735 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:36.916378 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c05a3ac6-52cf-44f8-9dc5-0ec58a3684d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:36.917221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c05a3ac6-52cf-44f8-9dc5-0ec58a3684d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:36.929948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c05a3ac6-52cf-44f8-9dc5-0ec58a3684d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:36.932646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c05a3ac6-52cf-44f8-9dc5-0ec58a3684d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:36.933500 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c05a3ac6-52cf-44f8-9dc5-0ec58a3684d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:36.935984 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c05a3ac6-52cf-44f8-9dc5-0ec58a3684d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:36.937455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c05a3ac6-52cf-44f8-9dc5-0ec58a3684d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:36.939559 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c05a3ac6-52cf-44f8-9dc5-0ec58a3684d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:37.121401 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c05a3ac6-52cf-44f8-9dc5-0ec58a3684d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:37.134743 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c05a3ac6-52cf-44f8-9dc5-0ec58a3684d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:37.136941 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c05a3ac6-52cf-44f8-9dc5-0ec58a3684d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:37.145571 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c05a3ac6-52cf-44f8-9dc5-0ec58a3684d9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1735 microversion: 2.20 time: 0.233484 Sep 08 14:31:37.146193 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1492/2703] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:36 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1735 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:38.176056 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0426a1dd-96ee-4502-9e98-b97e6966a034 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:38.180378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0426a1dd-96ee-4502-9e98-b97e6966a034 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:38.184366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0426a1dd-96ee-4502-9e98-b97e6966a034 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:38.184955 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0426a1dd-96ee-4502-9e98-b97e6966a034 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:38.185655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0426a1dd-96ee-4502-9e98-b97e6966a034 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:38.186826 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0426a1dd-96ee-4502-9e98-b97e6966a034 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:38.188106 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0426a1dd-96ee-4502-9e98-b97e6966a034 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:38.189057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0426a1dd-96ee-4502-9e98-b97e6966a034 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:38.354590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0426a1dd-96ee-4502-9e98-b97e6966a034 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:38.364619 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0426a1dd-96ee-4502-9e98-b97e6966a034 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:38.366162 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-0426a1dd-96ee-4502-9e98-b97e6966a034 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:38.368798 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0426a1dd-96ee-4502-9e98-b97e6966a034 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1735 microversion: 2.20 time: 0.196600 Sep 08 14:31:38.369342 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1212/2704] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:38 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1735 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:38.513120 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-905ce8e0-548d-481f-8f34-8fd3c3bf1284 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "4fc84b59-b08f-4b94-839c-0dbb61139175"}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:31:38.517294 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-905ce8e0-548d-481f-8f34-8fd3c3bf1284 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:38.525166 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-905ce8e0-548d-481f-8f34-8fd3c3bf1284 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:38.527230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-905ce8e0-548d-481f-8f34-8fd3c3bf1284 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:38.527779 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-905ce8e0-548d-481f-8f34-8fd3c3bf1284 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:38.528579 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-905ce8e0-548d-481f-8f34-8fd3c3bf1284 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:38.529104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-905ce8e0-548d-481f-8f34-8fd3c3bf1284 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:38.529990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-905ce8e0-548d-481f-8f34-8fd3c3bf1284 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:39.397602 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f0568757-6e7c-4be2-b8fa-4f721ed9f888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:39.398306 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f0568757-6e7c-4be2-b8fa-4f721ed9f888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:39.404966 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f0568757-6e7c-4be2-b8fa-4f721ed9f888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:39.405656 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f0568757-6e7c-4be2-b8fa-4f721ed9f888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:39.406344 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f0568757-6e7c-4be2-b8fa-4f721ed9f888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:39.407032 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f0568757-6e7c-4be2-b8fa-4f721ed9f888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:39.407655 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f0568757-6e7c-4be2-b8fa-4f721ed9f888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:39.408295 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f0568757-6e7c-4be2-b8fa-4f721ed9f888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:39.577817 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-905ce8e0-548d-481f-8f34-8fd3c3bf1284 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "POST /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.069317 Sep 08 14:31:39.578963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1493/2705] 10.222.0.25 () {60 vars in 1323 bytes} [Thu Sep 8 14:31:38 2022] POST /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f/os-volume_attachments => generated 194 bytes in 1072 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:31:39.622530 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f0568757-6e7c-4be2-b8fa-4f721ed9f888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:39.636047 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f0568757-6e7c-4be2-b8fa-4f721ed9f888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:39.637717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f0568757-6e7c-4be2-b8fa-4f721ed9f888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:39.640812 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f0568757-6e7c-4be2-b8fa-4f721ed9f888 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1735 microversion: 2.20 time: 0.246567 Sep 08 14:31:39.642085 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1213/2706] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:39 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1735 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:40.673887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3df1771e-a0a6-401d-ad79-53175a7c6b02 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:40.673887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3df1771e-a0a6-401d-ad79-53175a7c6b02 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:40.680259 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3df1771e-a0a6-401d-ad79-53175a7c6b02 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:40.680816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3df1771e-a0a6-401d-ad79-53175a7c6b02 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:40.685887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3df1771e-a0a6-401d-ad79-53175a7c6b02 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:40.685887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3df1771e-a0a6-401d-ad79-53175a7c6b02 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:40.685887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3df1771e-a0a6-401d-ad79-53175a7c6b02 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:40.685887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3df1771e-a0a6-401d-ad79-53175a7c6b02 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:40.821192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3df1771e-a0a6-401d-ad79-53175a7c6b02 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:40.834154 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3df1771e-a0a6-401d-ad79-53175a7c6b02 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:40.835736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3df1771e-a0a6-401d-ad79-53175a7c6b02 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:40.837207 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3df1771e-a0a6-401d-ad79-53175a7c6b02 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1735 microversion: 2.20 time: 0.171638 Sep 08 14:31:40.837908 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1494/2707] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:40 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1735 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:41.862654 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6d8bc80f-b061-45b2-bb51-3ea7aeaab0be tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:41.863542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6d8bc80f-b061-45b2-bb51-3ea7aeaab0be tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:41.870152 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d8bc80f-b061-45b2-bb51-3ea7aeaab0be tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:41.870560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d8bc80f-b061-45b2-bb51-3ea7aeaab0be tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:41.870992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d8bc80f-b061-45b2-bb51-3ea7aeaab0be tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:41.871687 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d8bc80f-b061-45b2-bb51-3ea7aeaab0be tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:41.872133 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d8bc80f-b061-45b2-bb51-3ea7aeaab0be tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:41.872582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d8bc80f-b061-45b2-bb51-3ea7aeaab0be tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:42.021898 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6d8bc80f-b061-45b2-bb51-3ea7aeaab0be tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:42.027157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6d8bc80f-b061-45b2-bb51-3ea7aeaab0be tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:42.028637 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6d8bc80f-b061-45b2-bb51-3ea7aeaab0be tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:42.030015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6d8bc80f-b061-45b2-bb51-3ea7aeaab0be tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1717 microversion: 2.20 time: 0.171986 Sep 08 14:31:42.030616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1214/2708] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:41 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1717 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:43.055900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-af607212-ad46-40be-90c9-49281330a971 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:43.056501 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-af607212-ad46-40be-90c9-49281330a971 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:43.062929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af607212-ad46-40be-90c9-49281330a971 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:43.063406 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af607212-ad46-40be-90c9-49281330a971 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:43.063838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af607212-ad46-40be-90c9-49281330a971 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:43.064516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af607212-ad46-40be-90c9-49281330a971 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:43.065539 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af607212-ad46-40be-90c9-49281330a971 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:43.065969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-af607212-ad46-40be-90c9-49281330a971 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:43.201415 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-af607212-ad46-40be-90c9-49281330a971 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:43.204526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-af607212-ad46-40be-90c9-49281330a971 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:43.206603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-af607212-ad46-40be-90c9-49281330a971 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:43.207758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-af607212-ad46-40be-90c9-49281330a971 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1717 microversion: 2.20 time: 0.155735 Sep 08 14:31:43.208319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1495/2709] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:43 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1717 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:44.233391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a1895cce-af74-442f-b034-215245fb81bb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:44.233967 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a1895cce-af74-442f-b034-215245fb81bb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:44.236347 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=54,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:44.241062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1895cce-af74-442f-b034-215245fb81bb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:44.241542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1895cce-af74-442f-b034-215245fb81bb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:44.248847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1895cce-af74-442f-b034-215245fb81bb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:44.248847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1895cce-af74-442f-b034-215245fb81bb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:44.248847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1895cce-af74-442f-b034-215245fb81bb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:44.248847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a1895cce-af74-442f-b034-215245fb81bb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:44.374034 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a1895cce-af74-442f-b034-215245fb81bb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:44.384735 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a1895cce-af74-442f-b034-215245fb81bb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:44.386965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a1895cce-af74-442f-b034-215245fb81bb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:44.388144 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a1895cce-af74-442f-b034-215245fb81bb tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1717 microversion: 2.20 time: 0.158965 Sep 08 14:31:44.388701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1215/2710] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:44 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1717 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:45.414630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e620bdf7-1a3e-49c7-86a2-b4068f8fe1c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:45.415230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e620bdf7-1a3e-49c7-86a2-b4068f8fe1c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:45.421548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e620bdf7-1a3e-49c7-86a2-b4068f8fe1c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:45.422036 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e620bdf7-1a3e-49c7-86a2-b4068f8fe1c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:45.422517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e620bdf7-1a3e-49c7-86a2-b4068f8fe1c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:45.423206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e620bdf7-1a3e-49c7-86a2-b4068f8fe1c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:45.423661 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e620bdf7-1a3e-49c7-86a2-b4068f8fe1c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:45.424107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e620bdf7-1a3e-49c7-86a2-b4068f8fe1c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:45.556891 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e620bdf7-1a3e-49c7-86a2-b4068f8fe1c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:45.566217 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e620bdf7-1a3e-49c7-86a2-b4068f8fe1c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:45.567252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e620bdf7-1a3e-49c7-86a2-b4068f8fe1c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:45.568405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e620bdf7-1a3e-49c7-86a2-b4068f8fe1c5 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1717 microversion: 2.20 time: 0.157600 Sep 08 14:31:45.569016 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1496/2711] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:45 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1717 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:46.594503 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-52405f3f-23c9-4f3f-8091-99df544a00ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:46.595097 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-52405f3f-23c9-4f3f-8091-99df544a00ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:46.602183 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-52405f3f-23c9-4f3f-8091-99df544a00ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:46.602669 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-52405f3f-23c9-4f3f-8091-99df544a00ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:46.603099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-52405f3f-23c9-4f3f-8091-99df544a00ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:46.603798 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-52405f3f-23c9-4f3f-8091-99df544a00ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:46.604247 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-52405f3f-23c9-4f3f-8091-99df544a00ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:46.604672 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-52405f3f-23c9-4f3f-8091-99df544a00ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:46.624139 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-6852d12c-1cff-4e7b-bc26-c80d15582ff4 req-f90c28ad-91f7-4559-964e-28a8df7ec63c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b0741665-d24b-4c17-93c7-90bcfff633b9", "name": "network-vif-unplugged", "status": "completed", "tag": "67fe80b7-8ae7-46be-8809-bd4958623c19"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:31:46.635516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-6852d12c-1cff-4e7b-bc26-c80d15582ff4 req-f90c28ad-91f7-4559-964e-28a8df7ec63c service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:46.636027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-6852d12c-1cff-4e7b-bc26-c80d15582ff4 req-f90c28ad-91f7-4559-964e-28a8df7ec63c service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:46.636452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-6852d12c-1cff-4e7b-bc26-c80d15582ff4 req-f90c28ad-91f7-4559-964e-28a8df7ec63c service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:46.650396 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-6852d12c-1cff-4e7b-bc26-c80d15582ff4 req-f90c28ad-91f7-4559-964e-28a8df7ec63c service nova] Creating event network-vif-unplugged:67fe80b7-8ae7-46be-8809-bd4958623c19 for instance b0741665-d24b-4c17-93c7-90bcfff633b9 on hv-cinder-85646 Sep 08 14:31:46.655978 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-6852d12c-1cff-4e7b-bc26-c80d15582ff4 req-f90c28ad-91f7-4559-964e-28a8df7ec63c service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.035863 Sep 08 14:31:46.656586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1497/2712] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:31:46 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:31:46.758515 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-52405f3f-23c9-4f3f-8091-99df544a00ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:46.766933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-52405f3f-23c9-4f3f-8091-99df544a00ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:46.769114 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-52405f3f-23c9-4f3f-8091-99df544a00ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:46.770336 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-52405f3f-23c9-4f3f-8091-99df544a00ba tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1717 microversion: 2.20 time: 0.179898 Sep 08 14:31:46.770897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1216/2713] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:46 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1717 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:47.796014 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bbe7ee9a-0cea-4af0-8491-ad72099780c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:47.797284 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bbe7ee9a-0cea-4af0-8491-ad72099780c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:47.804986 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bbe7ee9a-0cea-4af0-8491-ad72099780c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:47.805521 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bbe7ee9a-0cea-4af0-8491-ad72099780c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:47.805960 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bbe7ee9a-0cea-4af0-8491-ad72099780c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:47.806673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bbe7ee9a-0cea-4af0-8491-ad72099780c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:47.807125 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bbe7ee9a-0cea-4af0-8491-ad72099780c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:47.807546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bbe7ee9a-0cea-4af0-8491-ad72099780c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:47.942082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bbe7ee9a-0cea-4af0-8491-ad72099780c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:47.950452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bbe7ee9a-0cea-4af0-8491-ad72099780c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:47.951881 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bbe7ee9a-0cea-4af0-8491-ad72099780c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:47.953036 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bbe7ee9a-0cea-4af0-8491-ad72099780c4 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1720 microversion: 2.20 time: 0.161235 Sep 08 14:31:47.953596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1498/2714] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:47 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1720 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:47.978161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8f55e650-a625-4e7d-9493-f4b482183eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9b071f20-5c69-4dbd-9340-0561b0b98119"}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:31:47.980485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8f55e650-a625-4e7d-9493-f4b482183eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:47.987374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f55e650-a625-4e7d-9493-f4b482183eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:47.987854 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f55e650-a625-4e7d-9493-f4b482183eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:47.988289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f55e650-a625-4e7d-9493-f4b482183eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:47.988979 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f55e650-a625-4e7d-9493-f4b482183eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:47.989428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f55e650-a625-4e7d-9493-f4b482183eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:47.989850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f55e650-a625-4e7d-9493-f4b482183eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:48.208698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cc407594-10e5-458a-82e8-d54f1a280876 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:31:48.214810 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cc407594-10e5-458a-82e8-d54f1a280876 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:48.215444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cc407594-10e5-458a-82e8-d54f1a280876 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:48.215881 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cc407594-10e5-458a-82e8-d54f1a280876 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:48.283099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cc407594-10e5-458a-82e8-d54f1a280876 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Going to try to stop instance {{(pid=127601) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 08 14:31:48.464394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cc407594-10e5-458a-82e8-d54f1a280876 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "POST /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f/action" status: 202 len: 0 microversion: 2.1 time: 0.256575 Sep 08 14:31:48.464394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1499/2715] 10.222.0.25 () {60 vars in 1278 bytes} [Thu Sep 8 14:31:48 2022] POST /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f/action => generated 0 bytes in 258 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 08 14:31:48.470674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c850e117-a2ef-4291-817f-c11ad64531fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:48.471428 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c850e117-a2ef-4291-817f-c11ad64531fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:48.477633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c850e117-a2ef-4291-817f-c11ad64531fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:48.478114 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c850e117-a2ef-4291-817f-c11ad64531fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:48.478580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c850e117-a2ef-4291-817f-c11ad64531fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:48.479271 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c850e117-a2ef-4291-817f-c11ad64531fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:48.479736 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c850e117-a2ef-4291-817f-c11ad64531fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:48.481775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c850e117-a2ef-4291-817f-c11ad64531fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:48.495412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:48.617198 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c850e117-a2ef-4291-817f-c11ad64531fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:48.625311 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c850e117-a2ef-4291-817f-c11ad64531fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.159448 Sep 08 14:31:48.626400 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1500/2716] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:48 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:31:48.897246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8f55e650-a625-4e7d-9493-f4b482183eab tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "POST /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.922931 Sep 08 14:31:48.897811 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1217/2717] 10.222.0.25 () {62 vars in 1364 bytes} [Thu Sep 8 14:31:47 2022] POST /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9/os-volume_attachments => generated 188 bytes in 924 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 08 14:31:49.032704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-72392613-036a-4442-874d-feb6a96d7603 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:49.033242 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-72392613-036a-4442-874d-feb6a96d7603 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:49.041219 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-72392613-036a-4442-874d-feb6a96d7603 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:49.041774 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-72392613-036a-4442-874d-feb6a96d7603 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:49.042313 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-72392613-036a-4442-874d-feb6a96d7603 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:49.046098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-72392613-036a-4442-874d-feb6a96d7603 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:49.046098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-72392613-036a-4442-874d-feb6a96d7603 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:49.046098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-72392613-036a-4442-874d-feb6a96d7603 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:49.536849 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-72392613-036a-4442-874d-feb6a96d7603 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9/os-volume_attachments/9b071f20-5c69-4dbd-9340-0561b0b98119" status: 202 len: 0 microversion: 2.20 time: 0.508073 Sep 08 14:31:49.537581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1501/2718] 10.222.0.25 () {60 vars in 1457 bytes} [Thu Sep 8 14:31:49 2022] DELETE /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9/os-volume_attachments/9b071f20-5c69-4dbd-9340-0561b0b98119 => generated 0 bytes in 510 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:31:49.600549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f18fe478-631a-4881-b1dd-152d4e8bb07b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:31:49.601125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f18fe478-631a-4881-b1dd-152d4e8bb07b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:49.607634 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f18fe478-631a-4881-b1dd-152d4e8bb07b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:49.608102 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f18fe478-631a-4881-b1dd-152d4e8bb07b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:49.608525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f18fe478-631a-4881-b1dd-152d4e8bb07b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:49.609196 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f18fe478-631a-4881-b1dd-152d4e8bb07b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:49.609636 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f18fe478-631a-4881-b1dd-152d4e8bb07b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:49.610050 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f18fe478-631a-4881-b1dd-152d4e8bb07b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:49.649658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-157fe6ae-3eb0-4d6a-9864-e789a8e6e122 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:49.650330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-157fe6ae-3eb0-4d6a-9864-e789a8e6e122 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:49.651195 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f18fe478-631a-4881-b1dd-152d4e8bb07b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=127602) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 08 14:31:49.656308 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-157fe6ae-3eb0-4d6a-9864-e789a8e6e122 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:49.656799 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-157fe6ae-3eb0-4d6a-9864-e789a8e6e122 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:49.657387 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-157fe6ae-3eb0-4d6a-9864-e789a8e6e122 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:49.658084 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-157fe6ae-3eb0-4d6a-9864-e789a8e6e122 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:49.658578 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-157fe6ae-3eb0-4d6a-9864-e789a8e6e122 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:49.659007 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-157fe6ae-3eb0-4d6a-9864-e789a8e6e122 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:49.774509 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=92,nova_cell1:INSERT=3,nova_cell1:UPDATE=5 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:31:49.794792 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-157fe6ae-3eb0-4d6a-9864-e789a8e6e122 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:49.802900 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-157fe6ae-3eb0-4d6a-9864-e789a8e6e122 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.157173 Sep 08 14:31:49.803452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1502/2719] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:49 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:31:50.003303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f18fe478-631a-4881-b1dd-152d4e8bb07b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "POST /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9/action" status: 202 len: 0 microversion: 2.20 time: 0.406552 Sep 08 14:31:50.003842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1218/2720] 10.222.0.25 () {62 vars in 1319 bytes} [Thu Sep 8 14:31:49 2022] POST /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9/action => generated 0 bytes in 408 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:31:50.013526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-81519bd0-8356-4f46-a1ef-125e7aae9cfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:50.014829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-81519bd0-8356-4f46-a1ef-125e7aae9cfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:50.022423 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81519bd0-8356-4f46-a1ef-125e7aae9cfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:50.026370 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81519bd0-8356-4f46-a1ef-125e7aae9cfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:50.026370 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81519bd0-8356-4f46-a1ef-125e7aae9cfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:50.026370 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81519bd0-8356-4f46-a1ef-125e7aae9cfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:50.026370 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81519bd0-8356-4f46-a1ef-125e7aae9cfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:50.030362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81519bd0-8356-4f46-a1ef-125e7aae9cfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:50.164303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-81519bd0-8356-4f46-a1ef-125e7aae9cfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:50.172803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-81519bd0-8356-4f46-a1ef-125e7aae9cfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:50.174285 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-81519bd0-8356-4f46-a1ef-125e7aae9cfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:50.175487 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-81519bd0-8356-4f46-a1ef-125e7aae9cfe tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1668 microversion: 2.20 time: 0.166054 Sep 08 14:31:50.176047 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1503/2721] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:50 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1668 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:50.827693 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b752881a-8286-4771-a9a5-12e01717e816 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:50.828309 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b752881a-8286-4771-a9a5-12e01717e816 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:50.835304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b752881a-8286-4771-a9a5-12e01717e816 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:50.835800 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b752881a-8286-4771-a9a5-12e01717e816 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:50.836239 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b752881a-8286-4771-a9a5-12e01717e816 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:50.836941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b752881a-8286-4771-a9a5-12e01717e816 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:50.837396 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b752881a-8286-4771-a9a5-12e01717e816 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:50.837824 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b752881a-8286-4771-a9a5-12e01717e816 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:50.972822 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b752881a-8286-4771-a9a5-12e01717e816 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:50.981052 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b752881a-8286-4771-a9a5-12e01717e816 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.157941 Sep 08 14:31:50.982220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1219/2722] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:50 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:31:51.202470 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-61c2c614-c94b-4211-84aa-4ed7910569c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:51.203180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-61c2c614-c94b-4211-84aa-4ed7910569c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:51.213218 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61c2c614-c94b-4211-84aa-4ed7910569c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:51.213878 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61c2c614-c94b-4211-84aa-4ed7910569c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:51.214419 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61c2c614-c94b-4211-84aa-4ed7910569c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:51.215299 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61c2c614-c94b-4211-84aa-4ed7910569c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:51.215804 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61c2c614-c94b-4211-84aa-4ed7910569c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:51.220800 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61c2c614-c94b-4211-84aa-4ed7910569c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:51.358027 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-61c2c614-c94b-4211-84aa-4ed7910569c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:51.367240 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-61c2c614-c94b-4211-84aa-4ed7910569c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:51.368691 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-61c2c614-c94b-4211-84aa-4ed7910569c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:51.369875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-61c2c614-c94b-4211-84aa-4ed7910569c2 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1668 microversion: 2.20 time: 0.172297 Sep 08 14:31:51.370475 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1504/2723] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:51 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1668 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 14:31:52.005614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-22f993fe-9f89-4a0c-9a9f-b911d5a75d82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:52.006225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-22f993fe-9f89-4a0c-9a9f-b911d5a75d82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:52.013028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-22f993fe-9f89-4a0c-9a9f-b911d5a75d82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:52.013507 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-22f993fe-9f89-4a0c-9a9f-b911d5a75d82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:52.013941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-22f993fe-9f89-4a0c-9a9f-b911d5a75d82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:52.015119 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-22f993fe-9f89-4a0c-9a9f-b911d5a75d82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:52.015421 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-22f993fe-9f89-4a0c-9a9f-b911d5a75d82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:52.015839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-22f993fe-9f89-4a0c-9a9f-b911d5a75d82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:52.176846 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-22f993fe-9f89-4a0c-9a9f-b911d5a75d82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:52.186061 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-22f993fe-9f89-4a0c-9a9f-b911d5a75d82 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.184400 Sep 08 14:31:52.186739 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1220/2724] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:52 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:31:52.397124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-aff75271-cb4b-4a88-ad79-f29c1e1acab3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:52.397858 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-aff75271-cb4b-4a88-ad79-f29c1e1acab3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:52.405331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aff75271-cb4b-4a88-ad79-f29c1e1acab3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:52.405828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aff75271-cb4b-4a88-ad79-f29c1e1acab3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:52.406420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aff75271-cb4b-4a88-ad79-f29c1e1acab3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:52.407128 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aff75271-cb4b-4a88-ad79-f29c1e1acab3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:52.407766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aff75271-cb4b-4a88-ad79-f29c1e1acab3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:52.408261 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-aff75271-cb4b-4a88-ad79-f29c1e1acab3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:52.557917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-aff75271-cb4b-4a88-ad79-f29c1e1acab3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:52.578018 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-aff75271-cb4b-4a88-ad79-f29c1e1acab3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:52.580107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-aff75271-cb4b-4a88-ad79-f29c1e1acab3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:52.581393 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-aff75271-cb4b-4a88-ad79-f29c1e1acab3 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.188387 Sep 08 14:31:52.584684 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1505/2725] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:52 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:53.209639 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-29590d76-68b8-4d3e-a4f1-e4faf993c342 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:53.210326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-29590d76-68b8-4d3e-a4f1-e4faf993c342 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:53.220218 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29590d76-68b8-4d3e-a4f1-e4faf993c342 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:53.226538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29590d76-68b8-4d3e-a4f1-e4faf993c342 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:53.226538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29590d76-68b8-4d3e-a4f1-e4faf993c342 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:53.226538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29590d76-68b8-4d3e-a4f1-e4faf993c342 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:53.226538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29590d76-68b8-4d3e-a4f1-e4faf993c342 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:53.227185 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-29590d76-68b8-4d3e-a4f1-e4faf993c342 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:53.394389 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-29590d76-68b8-4d3e-a4f1-e4faf993c342 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:53.402802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-29590d76-68b8-4d3e-a4f1-e4faf993c342 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.197634 Sep 08 14:31:53.403572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1221/2726] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:53 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:31:53.613025 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e059f22f-5c09-4cab-a3b6-766aaa236336 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:53.613704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e059f22f-5c09-4cab-a3b6-766aaa236336 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:53.619959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e059f22f-5c09-4cab-a3b6-766aaa236336 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:53.620432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e059f22f-5c09-4cab-a3b6-766aaa236336 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:53.621082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e059f22f-5c09-4cab-a3b6-766aaa236336 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:53.621775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e059f22f-5c09-4cab-a3b6-766aaa236336 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:53.622256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e059f22f-5c09-4cab-a3b6-766aaa236336 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:53.622682 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e059f22f-5c09-4cab-a3b6-766aaa236336 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:53.785925 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e059f22f-5c09-4cab-a3b6-766aaa236336 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:53.792622 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e059f22f-5c09-4cab-a3b6-766aaa236336 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:53.794084 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e059f22f-5c09-4cab-a3b6-766aaa236336 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:53.795290 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e059f22f-5c09-4cab-a3b6-766aaa236336 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.186921 Sep 08 14:31:53.795857 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1506/2727] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:53 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:54.429231 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-18ebc028-28bc-45c3-958b-3c4c568d62e3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:54.429935 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-18ebc028-28bc-45c3-958b-3c4c568d62e3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:54.447188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18ebc028-28bc-45c3-958b-3c4c568d62e3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:54.448788 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18ebc028-28bc-45c3-958b-3c4c568d62e3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:54.452093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18ebc028-28bc-45c3-958b-3c4c568d62e3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:54.453694 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18ebc028-28bc-45c3-958b-3c4c568d62e3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:54.458657 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18ebc028-28bc-45c3-958b-3c4c568d62e3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:54.458657 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-18ebc028-28bc-45c3-958b-3c4c568d62e3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:54.661516 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-18ebc028-28bc-45c3-958b-3c4c568d62e3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:54.671315 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-18ebc028-28bc-45c3-958b-3c4c568d62e3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.247581 Sep 08 14:31:54.672874 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1222/2728] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:54 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:31:54.823623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-891d3ddf-e279-4bc0-b920-c147686bf4f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:54.823623 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-891d3ddf-e279-4bc0-b920-c147686bf4f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:54.828874 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-891d3ddf-e279-4bc0-b920-c147686bf4f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:54.829375 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-891d3ddf-e279-4bc0-b920-c147686bf4f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:54.829795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-891d3ddf-e279-4bc0-b920-c147686bf4f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:54.830542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-891d3ddf-e279-4bc0-b920-c147686bf4f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:54.830997 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-891d3ddf-e279-4bc0-b920-c147686bf4f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:54.831416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-891d3ddf-e279-4bc0-b920-c147686bf4f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:55.020642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-891d3ddf-e279-4bc0-b920-c147686bf4f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:55.034221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-891d3ddf-e279-4bc0-b920-c147686bf4f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:55.038121 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-891d3ddf-e279-4bc0-b920-c147686bf4f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:55.041681 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-891d3ddf-e279-4bc0-b920-c147686bf4f7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.223686 Sep 08 14:31:55.044227 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1507/2729] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:54 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:55.697989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-43271239-5b4a-4764-b483-fb75e47f75d9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:55.698638 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-43271239-5b4a-4764-b483-fb75e47f75d9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:55.705681 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-43271239-5b4a-4764-b483-fb75e47f75d9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:55.706103 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-43271239-5b4a-4764-b483-fb75e47f75d9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:55.706793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-43271239-5b4a-4764-b483-fb75e47f75d9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:55.707480 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-43271239-5b4a-4764-b483-fb75e47f75d9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:55.708439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-43271239-5b4a-4764-b483-fb75e47f75d9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:55.708867 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-43271239-5b4a-4764-b483-fb75e47f75d9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:55.858686 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-43271239-5b4a-4764-b483-fb75e47f75d9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:55.867563 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-43271239-5b4a-4764-b483-fb75e47f75d9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.173394 Sep 08 14:31:55.868125 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1223/2730] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:55 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:31:56.069468 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3d828b92-8503-4b84-b557-aa03a2ec9954 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:56.070022 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3d828b92-8503-4b84-b557-aa03a2ec9954 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:56.076463 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d828b92-8503-4b84-b557-aa03a2ec9954 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:56.077003 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d828b92-8503-4b84-b557-aa03a2ec9954 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:56.077450 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d828b92-8503-4b84-b557-aa03a2ec9954 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:56.078172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d828b92-8503-4b84-b557-aa03a2ec9954 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:56.078637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d828b92-8503-4b84-b557-aa03a2ec9954 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:56.079071 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3d828b92-8503-4b84-b557-aa03a2ec9954 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:56.213938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3d828b92-8503-4b84-b557-aa03a2ec9954 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:56.222014 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3d828b92-8503-4b84-b557-aa03a2ec9954 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:56.223494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3d828b92-8503-4b84-b557-aa03a2ec9954 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:56.224649 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3d828b92-8503-4b84-b557-aa03a2ec9954 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.159176 Sep 08 14:31:56.225210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1508/2731] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:56 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:56.893072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6afcaf0a-9332-45e2-8fca-567a99a23646 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:56.893612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6afcaf0a-9332-45e2-8fca-567a99a23646 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:56.900007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6afcaf0a-9332-45e2-8fca-567a99a23646 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:56.900484 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6afcaf0a-9332-45e2-8fca-567a99a23646 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:56.900911 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6afcaf0a-9332-45e2-8fca-567a99a23646 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:56.901620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6afcaf0a-9332-45e2-8fca-567a99a23646 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:56.902200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6afcaf0a-9332-45e2-8fca-567a99a23646 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:56.902622 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6afcaf0a-9332-45e2-8fca-567a99a23646 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:57.067804 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6afcaf0a-9332-45e2-8fca-567a99a23646 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:57.076112 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6afcaf0a-9332-45e2-8fca-567a99a23646 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.187805 Sep 08 14:31:57.076689 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1224/2732] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:56 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:31:57.250548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5a865ca8-50d5-4915-86c8-70f623dc0490 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:57.252684 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5a865ca8-50d5-4915-86c8-70f623dc0490 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:57.272996 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a865ca8-50d5-4915-86c8-70f623dc0490 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:57.273853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a865ca8-50d5-4915-86c8-70f623dc0490 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:57.274434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a865ca8-50d5-4915-86c8-70f623dc0490 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:57.275337 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a865ca8-50d5-4915-86c8-70f623dc0490 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:57.275893 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a865ca8-50d5-4915-86c8-70f623dc0490 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:57.276451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5a865ca8-50d5-4915-86c8-70f623dc0490 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:57.421932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5a865ca8-50d5-4915-86c8-70f623dc0490 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:57.430774 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5a865ca8-50d5-4915-86c8-70f623dc0490 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:57.432201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5a865ca8-50d5-4915-86c8-70f623dc0490 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:57.433346 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5a865ca8-50d5-4915-86c8-70f623dc0490 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.187082 Sep 08 14:31:57.433899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1509/2733] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:57 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:58.100493 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1f432b75-2432-4a98-945a-1b67792d8804 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:58.101234 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1f432b75-2432-4a98-945a-1b67792d8804 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:58.108228 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f432b75-2432-4a98-945a-1b67792d8804 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:58.108715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f432b75-2432-4a98-945a-1b67792d8804 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:58.109147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f432b75-2432-4a98-945a-1b67792d8804 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:58.110015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f432b75-2432-4a98-945a-1b67792d8804 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:58.110462 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f432b75-2432-4a98-945a-1b67792d8804 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:58.111007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1f432b75-2432-4a98-945a-1b67792d8804 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:58.266521 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1f432b75-2432-4a98-945a-1b67792d8804 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:58.275176 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1f432b75-2432-4a98-945a-1b67792d8804 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.179669 Sep 08 14:31:58.275745 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1225/2734] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:58 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:31:58.460447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3404eda4-c7a3-4583-9578-91656ed98d3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:58.461148 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3404eda4-c7a3-4583-9578-91656ed98d3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:58.468866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3404eda4-c7a3-4583-9578-91656ed98d3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:58.469340 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3404eda4-c7a3-4583-9578-91656ed98d3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:58.469775 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3404eda4-c7a3-4583-9578-91656ed98d3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:58.470494 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3404eda4-c7a3-4583-9578-91656ed98d3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:58.471591 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3404eda4-c7a3-4583-9578-91656ed98d3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:58.472018 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3404eda4-c7a3-4583-9578-91656ed98d3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:58.602939 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3404eda4-c7a3-4583-9578-91656ed98d3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:58.611041 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3404eda4-c7a3-4583-9578-91656ed98d3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:58.612482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3404eda4-c7a3-4583-9578-91656ed98d3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:58.613636 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3404eda4-c7a3-4583-9578-91656ed98d3b tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.159157 Sep 08 14:31:58.614216 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1510/2735] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:58 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:31:59.299949 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a9606fa8-2041-4d18-9405-92c50adf3215 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:59.300580 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a9606fa8-2041-4d18-9405-92c50adf3215 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:59.312632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a9606fa8-2041-4d18-9405-92c50adf3215 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:59.313124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a9606fa8-2041-4d18-9405-92c50adf3215 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:59.313561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a9606fa8-2041-4d18-9405-92c50adf3215 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:59.314301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a9606fa8-2041-4d18-9405-92c50adf3215 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:59.315578 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a9606fa8-2041-4d18-9405-92c50adf3215 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:59.316222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a9606fa8-2041-4d18-9405-92c50adf3215 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:59.464527 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a9606fa8-2041-4d18-9405-92c50adf3215 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:59.472848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a9606fa8-2041-4d18-9405-92c50adf3215 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.177499 Sep 08 14:31:59.473408 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1226/2736] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:31:59 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:31:59.639625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d14136e8-b6b2-4a95-b2dc-b1452a425c7f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:31:59.639625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d14136e8-b6b2-4a95-b2dc-b1452a425c7f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:31:59.646414 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d14136e8-b6b2-4a95-b2dc-b1452a425c7f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:59.647066 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d14136e8-b6b2-4a95-b2dc-b1452a425c7f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:59.647724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d14136e8-b6b2-4a95-b2dc-b1452a425c7f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:59.648759 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d14136e8-b6b2-4a95-b2dc-b1452a425c7f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:31:59.649455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d14136e8-b6b2-4a95-b2dc-b1452a425c7f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:31:59.650097 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d14136e8-b6b2-4a95-b2dc-b1452a425c7f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:31:59.809503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d14136e8-b6b2-4a95-b2dc-b1452a425c7f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:59.819112 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d14136e8-b6b2-4a95-b2dc-b1452a425c7f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:59.820825 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d14136e8-b6b2-4a95-b2dc-b1452a425c7f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:31:59.822073 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d14136e8-b6b2-4a95-b2dc-b1452a425c7f tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.186951 Sep 08 14:31:59.823516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1511/2737] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:31:59 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:32:00.497367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-abcba0a8-cac1-47f7-95fa-a834dbf47df6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:00.498023 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-abcba0a8-cac1-47f7-95fa-a834dbf47df6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:00.505124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abcba0a8-cac1-47f7-95fa-a834dbf47df6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:00.508040 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abcba0a8-cac1-47f7-95fa-a834dbf47df6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:00.508040 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abcba0a8-cac1-47f7-95fa-a834dbf47df6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:00.508040 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abcba0a8-cac1-47f7-95fa-a834dbf47df6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:00.508040 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abcba0a8-cac1-47f7-95fa-a834dbf47df6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:00.508493 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-abcba0a8-cac1-47f7-95fa-a834dbf47df6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:00.655575 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-abcba0a8-cac1-47f7-95fa-a834dbf47df6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:00.665167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-abcba0a8-cac1-47f7-95fa-a834dbf47df6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.171956 Sep 08 14:32:00.682790 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1227/2738] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:32:00 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:32:00.855228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-61f97648-7bdc-4a82-b8a0-603c88417c85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:00.856107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-61f97648-7bdc-4a82-b8a0-603c88417c85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:00.859945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=46 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:00.864560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61f97648-7bdc-4a82-b8a0-603c88417c85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:00.865119 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61f97648-7bdc-4a82-b8a0-603c88417c85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:00.865559 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61f97648-7bdc-4a82-b8a0-603c88417c85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:00.866584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61f97648-7bdc-4a82-b8a0-603c88417c85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:00.867252 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61f97648-7bdc-4a82-b8a0-603c88417c85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:00.867908 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-61f97648-7bdc-4a82-b8a0-603c88417c85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:01.039817 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-61f97648-7bdc-4a82-b8a0-603c88417c85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:01.048313 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-61f97648-7bdc-4a82-b8a0-603c88417c85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:01.049724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-61f97648-7bdc-4a82-b8a0-603c88417c85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:01.050912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-61f97648-7bdc-4a82-b8a0-603c88417c85 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.201256 Sep 08 14:32:01.051510 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1512/2739] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:32:00 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:32:01.691399 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-adf48841-d389-4d8d-8c31-26f6a4dcf9c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:01.692006 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-adf48841-d389-4d8d-8c31-26f6a4dcf9c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:01.698560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-adf48841-d389-4d8d-8c31-26f6a4dcf9c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:01.699045 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-adf48841-d389-4d8d-8c31-26f6a4dcf9c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:01.699508 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-adf48841-d389-4d8d-8c31-26f6a4dcf9c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:01.700191 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-adf48841-d389-4d8d-8c31-26f6a4dcf9c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:01.700640 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-adf48841-d389-4d8d-8c31-26f6a4dcf9c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:01.701065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-adf48841-d389-4d8d-8c31-26f6a4dcf9c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:01.831776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-adf48841-d389-4d8d-8c31-26f6a4dcf9c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:01.839966 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-adf48841-d389-4d8d-8c31-26f6a4dcf9c4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.153116 Sep 08 14:32:01.840913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1228/2740] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:32:01 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:32:02.079228 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-75d95d25-29c2-459d-a7b4-6fab4f86245e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:02.079860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-75d95d25-29c2-459d-a7b4-6fab4f86245e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:02.087497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75d95d25-29c2-459d-a7b4-6fab4f86245e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:02.087993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75d95d25-29c2-459d-a7b4-6fab4f86245e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:02.088712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75d95d25-29c2-459d-a7b4-6fab4f86245e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:02.089509 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75d95d25-29c2-459d-a7b4-6fab4f86245e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:02.089980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75d95d25-29c2-459d-a7b4-6fab4f86245e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:02.090550 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-75d95d25-29c2-459d-a7b4-6fab4f86245e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:02.250163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-75d95d25-29c2-459d-a7b4-6fab4f86245e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:02.259847 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-75d95d25-29c2-459d-a7b4-6fab4f86245e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:02.261752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-75d95d25-29c2-459d-a7b4-6fab4f86245e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:02.263485 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-75d95d25-29c2-459d-a7b4-6fab4f86245e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.188201 Sep 08 14:32:02.281731 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1513/2741] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:32:02 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 14:32:02.864359 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a067518f-a57b-4e7d-8b1b-067d71cd25b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:02.865912 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a067518f-a57b-4e7d-8b1b-067d71cd25b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:02.872917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a067518f-a57b-4e7d-8b1b-067d71cd25b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:02.873399 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a067518f-a57b-4e7d-8b1b-067d71cd25b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:02.873828 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a067518f-a57b-4e7d-8b1b-067d71cd25b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:02.874579 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a067518f-a57b-4e7d-8b1b-067d71cd25b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:02.875037 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a067518f-a57b-4e7d-8b1b-067d71cd25b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:02.875491 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a067518f-a57b-4e7d-8b1b-067d71cd25b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:03.024553 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a067518f-a57b-4e7d-8b1b-067d71cd25b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:03.034041 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a067518f-a57b-4e7d-8b1b-067d71cd25b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.173670 Sep 08 14:32:03.034935 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1229/2742] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:32:02 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:32:03.290165 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ef9e4f15-94f1-4b56-92f2-6c47b6068163 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:03.290866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ef9e4f15-94f1-4b56-92f2-6c47b6068163 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:03.297406 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef9e4f15-94f1-4b56-92f2-6c47b6068163 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:03.298237 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef9e4f15-94f1-4b56-92f2-6c47b6068163 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:03.298926 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef9e4f15-94f1-4b56-92f2-6c47b6068163 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:03.299872 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef9e4f15-94f1-4b56-92f2-6c47b6068163 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:03.300984 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef9e4f15-94f1-4b56-92f2-6c47b6068163 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:03.301409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-ef9e4f15-94f1-4b56-92f2-6c47b6068163 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:03.322580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:03.435343 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ef9e4f15-94f1-4b56-92f2-6c47b6068163 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:03.443912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ef9e4f15-94f1-4b56-92f2-6c47b6068163 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:03.445353 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ef9e4f15-94f1-4b56-92f2-6c47b6068163 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:03.446533 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ef9e4f15-94f1-4b56-92f2-6c47b6068163 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.160435 Sep 08 14:32:03.447146 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1514/2743] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:32:03 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:32:04.061269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-b89d9466-b1ca-40c0-937f-97e50486f479 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:04.061887 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-b89d9466-b1ca-40c0-937f-97e50486f479 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:04.068918 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b89d9466-b1ca-40c0-937f-97e50486f479 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:04.069396 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b89d9466-b1ca-40c0-937f-97e50486f479 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:04.069838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b89d9466-b1ca-40c0-937f-97e50486f479 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:04.070529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b89d9466-b1ca-40c0-937f-97e50486f479 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:04.070975 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b89d9466-b1ca-40c0-937f-97e50486f479 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:04.071391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-b89d9466-b1ca-40c0-937f-97e50486f479 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:04.233282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-b89d9466-b1ca-40c0-937f-97e50486f479 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:04.242665 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-b89d9466-b1ca-40c0-937f-97e50486f479 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.185009 Sep 08 14:32:04.243840 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1230/2744] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:32:04 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:32:04.487762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-cca984a5-dfe8-442c-9f94-d3ac09ac0b76 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:04.490568 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-cca984a5-dfe8-442c-9f94-d3ac09ac0b76 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:04.501511 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cca984a5-dfe8-442c-9f94-d3ac09ac0b76 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:04.502057 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cca984a5-dfe8-442c-9f94-d3ac09ac0b76 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:04.504244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cca984a5-dfe8-442c-9f94-d3ac09ac0b76 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:04.504244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cca984a5-dfe8-442c-9f94-d3ac09ac0b76 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:04.504244 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cca984a5-dfe8-442c-9f94-d3ac09ac0b76 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:04.504698 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-cca984a5-dfe8-442c-9f94-d3ac09ac0b76 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:04.782548 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cca984a5-dfe8-442c-9f94-d3ac09ac0b76 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:04.793505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cca984a5-dfe8-442c-9f94-d3ac09ac0b76 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:04.795379 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-cca984a5-dfe8-442c-9f94-d3ac09ac0b76 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:04.796764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-cca984a5-dfe8-442c-9f94-d3ac09ac0b76 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.321021 Sep 08 14:32:04.797546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1515/2745] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:32:04 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:32:05.311609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6d2505a4-0a7c-4fcb-ab76-c1b2d36938e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:05.311609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6d2505a4-0a7c-4fcb-ab76-c1b2d36938e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:05.311609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d2505a4-0a7c-4fcb-ab76-c1b2d36938e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:05.311609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d2505a4-0a7c-4fcb-ab76-c1b2d36938e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:05.312609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d2505a4-0a7c-4fcb-ab76-c1b2d36938e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:05.312609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d2505a4-0a7c-4fcb-ab76-c1b2d36938e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:05.312609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d2505a4-0a7c-4fcb-ab76-c1b2d36938e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:05.312609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6d2505a4-0a7c-4fcb-ab76-c1b2d36938e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:05.452043 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6d2505a4-0a7c-4fcb-ab76-c1b2d36938e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:05.461688 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6d2505a4-0a7c-4fcb-ab76-c1b2d36938e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.195942 Sep 08 14:32:05.474753 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1231/2746] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:32:05 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:32:05.824180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-48189cb3-947f-45bc-bec2-e26dc327a544 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:05.824778 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-48189cb3-947f-45bc-bec2-e26dc327a544 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:05.831459 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-48189cb3-947f-45bc-bec2-e26dc327a544 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:05.831943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-48189cb3-947f-45bc-bec2-e26dc327a544 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:05.832379 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-48189cb3-947f-45bc-bec2-e26dc327a544 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:05.833066 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-48189cb3-947f-45bc-bec2-e26dc327a544 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:05.833513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-48189cb3-947f-45bc-bec2-e26dc327a544 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:05.833933 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-48189cb3-947f-45bc-bec2-e26dc327a544 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:06.122123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-48189cb3-947f-45bc-bec2-e26dc327a544 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:06.130663 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-48189cb3-947f-45bc-bec2-e26dc327a544 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:06.132153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-48189cb3-947f-45bc-bec2-e26dc327a544 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:06.133317 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-48189cb3-947f-45bc-bec2-e26dc327a544 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.313262 Sep 08 14:32:06.133874 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1516/2747] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:32:05 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 14:32:06.486718 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-8f399d5c-eec4-453e-a973-46f48784cb0d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:06.487315 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-8f399d5c-eec4-453e-a973-46f48784cb0d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:06.493802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f399d5c-eec4-453e-a973-46f48784cb0d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:06.494309 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f399d5c-eec4-453e-a973-46f48784cb0d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:06.494827 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f399d5c-eec4-453e-a973-46f48784cb0d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:06.495417 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f399d5c-eec4-453e-a973-46f48784cb0d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:06.495859 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f399d5c-eec4-453e-a973-46f48784cb0d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:06.496277 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-8f399d5c-eec4-453e-a973-46f48784cb0d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:06.648372 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-8f399d5c-eec4-453e-a973-46f48784cb0d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:06.657148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-8f399d5c-eec4-453e-a973-46f48784cb0d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.174366 Sep 08 14:32:06.657729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1232/2748] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:32:06 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:32:07.161952 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-91a15b89-2632-4308-af59-c6f1d79d6fc6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:07.162976 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-91a15b89-2632-4308-af59-c6f1d79d6fc6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:07.173629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91a15b89-2632-4308-af59-c6f1d79d6fc6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:07.174472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91a15b89-2632-4308-af59-c6f1d79d6fc6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:07.175223 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91a15b89-2632-4308-af59-c6f1d79d6fc6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:07.176293 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91a15b89-2632-4308-af59-c6f1d79d6fc6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:07.177078 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91a15b89-2632-4308-af59-c6f1d79d6fc6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:07.177799 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-91a15b89-2632-4308-af59-c6f1d79d6fc6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:07.318467 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-91a15b89-2632-4308-af59-c6f1d79d6fc6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:07.326680 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-91a15b89-2632-4308-af59-c6f1d79d6fc6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:07.328110 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-91a15b89-2632-4308-af59-c6f1d79d6fc6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:07.329334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-91a15b89-2632-4308-af59-c6f1d79d6fc6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.173953 Sep 08 14:32:07.329879 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1517/2749] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:32:07 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:32:07.686455 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9b6c5278-56f8-4207-8d03-534971a7e75c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:07.687071 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9b6c5278-56f8-4207-8d03-534971a7e75c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:07.694311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9b6c5278-56f8-4207-8d03-534971a7e75c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:07.701146 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9b6c5278-56f8-4207-8d03-534971a7e75c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:07.701146 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9b6c5278-56f8-4207-8d03-534971a7e75c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:07.701146 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9b6c5278-56f8-4207-8d03-534971a7e75c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:07.701146 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9b6c5278-56f8-4207-8d03-534971a7e75c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:07.701579 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9b6c5278-56f8-4207-8d03-534971a7e75c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:07.829244 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9b6c5278-56f8-4207-8d03-534971a7e75c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:07.838182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9b6c5278-56f8-4207-8d03-534971a7e75c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.155748 Sep 08 14:32:07.838858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1233/2750] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:32:07 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:32:08.356765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-fb89dc7b-0e8f-4263-a2a5-95bb162c5a5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:08.357376 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-fb89dc7b-0e8f-4263-a2a5-95bb162c5a5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:08.367212 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb89dc7b-0e8f-4263-a2a5-95bb162c5a5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:08.369412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb89dc7b-0e8f-4263-a2a5-95bb162c5a5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:08.369412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb89dc7b-0e8f-4263-a2a5-95bb162c5a5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:08.369412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb89dc7b-0e8f-4263-a2a5-95bb162c5a5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:08.369412 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb89dc7b-0e8f-4263-a2a5-95bb162c5a5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:08.370067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-fb89dc7b-0e8f-4263-a2a5-95bb162c5a5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:08.543453 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fb89dc7b-0e8f-4263-a2a5-95bb162c5a5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:08.552585 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fb89dc7b-0e8f-4263-a2a5-95bb162c5a5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:08.554033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-fb89dc7b-0e8f-4263-a2a5-95bb162c5a5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:08.555325 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-fb89dc7b-0e8f-4263-a2a5-95bb162c5a5e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.202489 Sep 08 14:32:08.556003 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1518/2751] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:32:08 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:32:08.862372 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a5f728ff-78ac-41da-b8c8-b57e057364a0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:08.863278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a5f728ff-78ac-41da-b8c8-b57e057364a0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:08.870370 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5f728ff-78ac-41da-b8c8-b57e057364a0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:08.875345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5f728ff-78ac-41da-b8c8-b57e057364a0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:08.875345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5f728ff-78ac-41da-b8c8-b57e057364a0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:08.875345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5f728ff-78ac-41da-b8c8-b57e057364a0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:08.875345 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5f728ff-78ac-41da-b8c8-b57e057364a0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:08.876252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a5f728ff-78ac-41da-b8c8-b57e057364a0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:09.057621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a5f728ff-78ac-41da-b8c8-b57e057364a0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:09.070463 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a5f728ff-78ac-41da-b8c8-b57e057364a0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.212620 Sep 08 14:32:09.071062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1234/2752] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:32:08 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:32:09.581827 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-43af8af3-b322-40df-ba53-db763ee2f0d6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:09.582465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-43af8af3-b322-40df-ba53-db763ee2f0d6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:09.589010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43af8af3-b322-40df-ba53-db763ee2f0d6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:09.589476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43af8af3-b322-40df-ba53-db763ee2f0d6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:09.590797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43af8af3-b322-40df-ba53-db763ee2f0d6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:09.590797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43af8af3-b322-40df-ba53-db763ee2f0d6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:09.591158 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43af8af3-b322-40df-ba53-db763ee2f0d6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:09.591589 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-43af8af3-b322-40df-ba53-db763ee2f0d6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:09.727814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-43af8af3-b322-40df-ba53-db763ee2f0d6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:09.736028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-43af8af3-b322-40df-ba53-db763ee2f0d6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:09.737451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-43af8af3-b322-40df-ba53-db763ee2f0d6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:09.738635 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-43af8af3-b322-40df-ba53-db763ee2f0d6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.160709 Sep 08 14:32:09.739176 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1519/2753] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:32:09 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:32:10.094340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6c24d17a-67dc-49e9-a4fe-e672aa99e400 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:10.094943 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6c24d17a-67dc-49e9-a4fe-e672aa99e400 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:10.101898 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c24d17a-67dc-49e9-a4fe-e672aa99e400 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:10.102416 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c24d17a-67dc-49e9-a4fe-e672aa99e400 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:10.102847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c24d17a-67dc-49e9-a4fe-e672aa99e400 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:10.103546 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c24d17a-67dc-49e9-a4fe-e672aa99e400 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:10.103974 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c24d17a-67dc-49e9-a4fe-e672aa99e400 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:10.104421 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c24d17a-67dc-49e9-a4fe-e672aa99e400 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:10.107311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:10.245723 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6c24d17a-67dc-49e9-a4fe-e672aa99e400 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:10.254038 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6c24d17a-67dc-49e9-a4fe-e672aa99e400 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.163502 Sep 08 14:32:10.259722 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1235/2754] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:32:10 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:32:10.763976 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3b8dbb77-144f-463e-a2f2-38fbf115a39e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:10.764586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3b8dbb77-144f-463e-a2f2-38fbf115a39e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:10.771912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3b8dbb77-144f-463e-a2f2-38fbf115a39e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:10.772409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3b8dbb77-144f-463e-a2f2-38fbf115a39e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:10.773455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3b8dbb77-144f-463e-a2f2-38fbf115a39e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:10.773595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3b8dbb77-144f-463e-a2f2-38fbf115a39e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:10.774017 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3b8dbb77-144f-463e-a2f2-38fbf115a39e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:10.774523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3b8dbb77-144f-463e-a2f2-38fbf115a39e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:10.931003 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3b8dbb77-144f-463e-a2f2-38fbf115a39e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:10.940241 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3b8dbb77-144f-463e-a2f2-38fbf115a39e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:10.941690 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3b8dbb77-144f-463e-a2f2-38fbf115a39e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:10.942980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3b8dbb77-144f-463e-a2f2-38fbf115a39e tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.182988 Sep 08 14:32:10.943530 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1520/2755] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:32:10 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:32:11.278354 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fde84a7a-74ab-457a-bf43-454a8567e575 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:11.278933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-fde84a7a-74ab-457a-bf43-454a8567e575 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:11.285376 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fde84a7a-74ab-457a-bf43-454a8567e575 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:11.285861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fde84a7a-74ab-457a-bf43-454a8567e575 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:11.286326 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fde84a7a-74ab-457a-bf43-454a8567e575 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:11.287038 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fde84a7a-74ab-457a-bf43-454a8567e575 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:11.287494 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fde84a7a-74ab-457a-bf43-454a8567e575 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:11.287915 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-fde84a7a-74ab-457a-bf43-454a8567e575 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:11.435742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-fde84a7a-74ab-457a-bf43-454a8567e575 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:11.445553 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fde84a7a-74ab-457a-bf43-454a8567e575 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1722 microversion: 2.1 time: 0.171137 Sep 08 14:32:11.447274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1236/2756] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:32:11 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1722 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:32:11.971481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-de88110f-9933-42e7-928d-00cf95f68678 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:11.972103 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-de88110f-9933-42e7-928d-00cf95f68678 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:11.979344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de88110f-9933-42e7-928d-00cf95f68678 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:11.979838 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de88110f-9933-42e7-928d-00cf95f68678 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:11.980279 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de88110f-9933-42e7-928d-00cf95f68678 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:11.980977 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de88110f-9933-42e7-928d-00cf95f68678 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:11.981439 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de88110f-9933-42e7-928d-00cf95f68678 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:11.981920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-de88110f-9933-42e7-928d-00cf95f68678 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:12.124491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-de88110f-9933-42e7-928d-00cf95f68678 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:12.132014 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-de88110f-9933-42e7-928d-00cf95f68678 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:12.133604 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-de88110f-9933-42e7-928d-00cf95f68678 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:12.134951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-de88110f-9933-42e7-928d-00cf95f68678 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.168521 Sep 08 14:32:12.135652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1521/2757] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:32:11 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:32:12.468972 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ff81d165-4459-4a91-a073-0ec19659deff tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:12.469574 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ff81d165-4459-4a91-a073-0ec19659deff tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:12.476360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff81d165-4459-4a91-a073-0ec19659deff tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:12.476873 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff81d165-4459-4a91-a073-0ec19659deff tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:12.477189 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff81d165-4459-4a91-a073-0ec19659deff tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:12.477905 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff81d165-4459-4a91-a073-0ec19659deff tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:12.478368 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff81d165-4459-4a91-a073-0ec19659deff tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:12.478784 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff81d165-4459-4a91-a073-0ec19659deff tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:12.621247 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ff81d165-4459-4a91-a073-0ec19659deff tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:12.630582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ff81d165-4459-4a91-a073-0ec19659deff tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1699 microversion: 2.1 time: 0.165557 Sep 08 14:32:12.631374 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1237/2758] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:32:12 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1699 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:32:12.656875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5bb399e3-225b-4f7b-a7d9-1a03b5ec401d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:32:12.663378 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5bb399e3-225b-4f7b-a7d9-1a03b5ec401d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:12.663937 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5bb399e3-225b-4f7b-a7d9-1a03b5ec401d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:12.664581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5bb399e3-225b-4f7b-a7d9-1a03b5ec401d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:12.692744 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5bb399e3-225b-4f7b-a7d9-1a03b5ec401d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Going to try to start instance {{(pid=127601) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 08 14:32:12.822397 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-00eb1299-327d-4ac6-8e3b-385c9309536d req-6d4accf5-cae6-46d7-a99c-c4592419ce1a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b0741665-d24b-4c17-93c7-90bcfff633b9", "name": "network-vif-plugged", "status": "completed", "tag": "67fe80b7-8ae7-46be-8809-bd4958623c19"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:32:12.832545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-00eb1299-327d-4ac6-8e3b-385c9309536d req-6d4accf5-cae6-46d7-a99c-c4592419ce1a service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:12.833510 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-00eb1299-327d-4ac6-8e3b-385c9309536d req-6d4accf5-cae6-46d7-a99c-c4592419ce1a service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:12.833510 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-00eb1299-327d-4ac6-8e3b-385c9309536d req-6d4accf5-cae6-46d7-a99c-c4592419ce1a service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:12.845093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-00eb1299-327d-4ac6-8e3b-385c9309536d req-6d4accf5-cae6-46d7-a99c-c4592419ce1a service nova] Creating event network-vif-plugged:67fe80b7-8ae7-46be-8809-bd4958623c19 for instance b0741665-d24b-4c17-93c7-90bcfff633b9 on hv-cinder-85646 Sep 08 14:32:12.850932 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-00eb1299-327d-4ac6-8e3b-385c9309536d req-6d4accf5-cae6-46d7-a99c-c4592419ce1a service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032423 Sep 08 14:32:12.852006 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1238/2759] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:32:12 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:32:12.869373 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5bb399e3-225b-4f7b-a7d9-1a03b5ec401d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "POST /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f/action" status: 202 len: 0 microversion: 2.1 time: 0.216630 Sep 08 14:32:12.870301 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1522/2760] 10.222.0.25 () {60 vars in 1278 bytes} [Thu Sep 8 14:32:12 2022] POST /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f/action => generated 0 bytes in 218 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:32:12.879253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d8f5e46f-ea49-45d8-8772-e8b4657c35eb tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:12.879973 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d8f5e46f-ea49-45d8-8772-e8b4657c35eb tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:12.886297 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d8f5e46f-ea49-45d8-8772-e8b4657c35eb tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:12.886782 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d8f5e46f-ea49-45d8-8772-e8b4657c35eb tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:12.887217 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d8f5e46f-ea49-45d8-8772-e8b4657c35eb tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:12.887909 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d8f5e46f-ea49-45d8-8772-e8b4657c35eb tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:12.888362 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d8f5e46f-ea49-45d8-8772-e8b4657c35eb tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:12.888783 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d8f5e46f-ea49-45d8-8772-e8b4657c35eb tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:13.063309 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d8f5e46f-ea49-45d8-8772-e8b4657c35eb tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:13.075067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d8f5e46f-ea49-45d8-8772-e8b4657c35eb tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1708 microversion: 2.1 time: 0.200196 Sep 08 14:32:13.075767 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1239/2761] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:32:12 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1708 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:32:13.163527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2e575aa9-c13f-407f-89c0-426f2b39266a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:13.164831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2e575aa9-c13f-407f-89c0-426f2b39266a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:13.172543 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e575aa9-c13f-407f-89c0-426f2b39266a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:13.173074 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e575aa9-c13f-407f-89c0-426f2b39266a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:13.174399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e575aa9-c13f-407f-89c0-426f2b39266a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:13.175391 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e575aa9-c13f-407f-89c0-426f2b39266a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:13.175588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e575aa9-c13f-407f-89c0-426f2b39266a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:13.176065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e575aa9-c13f-407f-89c0-426f2b39266a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:13.396831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2e575aa9-c13f-407f-89c0-426f2b39266a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:13.405567 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2e575aa9-c13f-407f-89c0-426f2b39266a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:13.408392 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2e575aa9-c13f-407f-89c0-426f2b39266a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:13.408714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2e575aa9-c13f-407f-89c0-426f2b39266a tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.251184 Sep 08 14:32:13.409642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1523/2762] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:32:13 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:32:14.099928 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-62433f39-2be2-44ef-be7a-effc11e4c52b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:14.100545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-62433f39-2be2-44ef-be7a-effc11e4c52b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:14.106821 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-62433f39-2be2-44ef-be7a-effc11e4c52b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:14.107300 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-62433f39-2be2-44ef-be7a-effc11e4c52b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:14.107728 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-62433f39-2be2-44ef-be7a-effc11e4c52b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:14.108412 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-62433f39-2be2-44ef-be7a-effc11e4c52b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:14.108861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-62433f39-2be2-44ef-be7a-effc11e4c52b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:14.109298 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-62433f39-2be2-44ef-be7a-effc11e4c52b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:14.245296 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-62433f39-2be2-44ef-be7a-effc11e4c52b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:14.254971 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-62433f39-2be2-44ef-be7a-effc11e4c52b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1708 microversion: 2.1 time: 0.159012 Sep 08 14:32:14.256057 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1240/2763] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:32:14 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1708 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:32:14.436634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-23685dd2-7252-410d-b5aa-de238a542c51 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:14.437399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-23685dd2-7252-410d-b5aa-de238a542c51 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:14.444631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23685dd2-7252-410d-b5aa-de238a542c51 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:14.445120 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23685dd2-7252-410d-b5aa-de238a542c51 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:14.445585 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23685dd2-7252-410d-b5aa-de238a542c51 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:14.446312 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23685dd2-7252-410d-b5aa-de238a542c51 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:14.446762 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23685dd2-7252-410d-b5aa-de238a542c51 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:14.447183 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-23685dd2-7252-410d-b5aa-de238a542c51 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:14.609788 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-23685dd2-7252-410d-b5aa-de238a542c51 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:14.618648 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-23685dd2-7252-410d-b5aa-de238a542c51 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:14.620085 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-23685dd2-7252-410d-b5aa-de238a542c51 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:14.621248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-23685dd2-7252-410d-b5aa-de238a542c51 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.188604 Sep 08 14:32:14.621810 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1524/2764] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:32:14 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:32:15.278839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bb9dd36b-89eb-453b-8ef2-c568504429c5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:15.279758 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bb9dd36b-89eb-453b-8ef2-c568504429c5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:15.282258 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=56,nova_api:UPDATE=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:15.287542 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bb9dd36b-89eb-453b-8ef2-c568504429c5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:15.288091 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bb9dd36b-89eb-453b-8ef2-c568504429c5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:15.288750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bb9dd36b-89eb-453b-8ef2-c568504429c5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:15.289452 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bb9dd36b-89eb-453b-8ef2-c568504429c5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:15.289900 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bb9dd36b-89eb-453b-8ef2-c568504429c5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:15.290354 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bb9dd36b-89eb-453b-8ef2-c568504429c5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:15.434254 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-bb9dd36b-89eb-453b-8ef2-c568504429c5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:15.443469 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bb9dd36b-89eb-453b-8ef2-c568504429c5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1708 microversion: 2.1 time: 0.168940 Sep 08 14:32:15.444033 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1241/2765] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:32:15 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1708 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:32:15.647378 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6b85ab96-4bf9-4254-bb14-83b54afd5766 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:15.648128 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6b85ab96-4bf9-4254-bb14-83b54afd5766 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:15.656786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b85ab96-4bf9-4254-bb14-83b54afd5766 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:15.657662 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b85ab96-4bf9-4254-bb14-83b54afd5766 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:15.658107 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b85ab96-4bf9-4254-bb14-83b54afd5766 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:15.659010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b85ab96-4bf9-4254-bb14-83b54afd5766 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:15.659362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b85ab96-4bf9-4254-bb14-83b54afd5766 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:15.659786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6b85ab96-4bf9-4254-bb14-83b54afd5766 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:15.798314 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6b85ab96-4bf9-4254-bb14-83b54afd5766 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:15.806927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6b85ab96-4bf9-4254-bb14-83b54afd5766 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:15.808364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6b85ab96-4bf9-4254-bb14-83b54afd5766 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:15.809566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6b85ab96-4bf9-4254-bb14-83b54afd5766 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.167298 Sep 08 14:32:15.810101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1525/2766] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:32:15 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:32:16.468705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f9d3be99-80d3-4341-b3b5-1409992e4b2f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:16.469352 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f9d3be99-80d3-4341-b3b5-1409992e4b2f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:16.476456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f9d3be99-80d3-4341-b3b5-1409992e4b2f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:16.476946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f9d3be99-80d3-4341-b3b5-1409992e4b2f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:16.477379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f9d3be99-80d3-4341-b3b5-1409992e4b2f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:16.478107 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f9d3be99-80d3-4341-b3b5-1409992e4b2f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:16.478590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f9d3be99-80d3-4341-b3b5-1409992e4b2f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:16.479014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f9d3be99-80d3-4341-b3b5-1409992e4b2f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:16.618645 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f9d3be99-80d3-4341-b3b5-1409992e4b2f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:16.626973 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f9d3be99-80d3-4341-b3b5-1409992e4b2f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1712 microversion: 2.1 time: 0.162635 Sep 08 14:32:16.628188 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1242/2767] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:32:16 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1712 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:32:16.835943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7ff11e53-5b37-43a5-ad6d-a893ab489a33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:16.836549 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7ff11e53-5b37-43a5-ad6d-a893ab489a33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:16.838654 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:32:16.838802 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:32:16 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 (ip 10.222.0.25) !!! Sep 08 14:32:16.846166 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7ff11e53-5b37-43a5-ad6d-a893ab489a33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:16.846671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7ff11e53-5b37-43a5-ad6d-a893ab489a33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:16.847104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7ff11e53-5b37-43a5-ad6d-a893ab489a33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:16.847795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7ff11e53-5b37-43a5-ad6d-a893ab489a33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:16.848242 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7ff11e53-5b37-43a5-ad6d-a893ab489a33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:16.848659 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7ff11e53-5b37-43a5-ad6d-a893ab489a33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:16.986245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7ff11e53-5b37-43a5-ad6d-a893ab489a33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:16.994554 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7ff11e53-5b37-43a5-ad6d-a893ab489a33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:16.995995 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7ff11e53-5b37-43a5-ad6d-a893ab489a33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:16.997170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7ff11e53-5b37-43a5-ad6d-a893ab489a33 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.165202 Sep 08 14:32:16.997729 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1526/2768] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:32:16 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:32:18.023436 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1233ed13-f085-48cb-8d66-765821429758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:18.023984 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1233ed13-f085-48cb-8d66-765821429758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:18.030927 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1233ed13-f085-48cb-8d66-765821429758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:18.031403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1233ed13-f085-48cb-8d66-765821429758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:18.031839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1233ed13-f085-48cb-8d66-765821429758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:18.032540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1233ed13-f085-48cb-8d66-765821429758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:18.032994 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1233ed13-f085-48cb-8d66-765821429758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:18.033417 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1233ed13-f085-48cb-8d66-765821429758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:18.173550 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1233ed13-f085-48cb-8d66-765821429758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:18.182015 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1233ed13-f085-48cb-8d66-765821429758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:18.183485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1233ed13-f085-48cb-8d66-765821429758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:18.184640 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1233ed13-f085-48cb-8d66-765821429758 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.165296 Sep 08 14:32:18.185200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1243/2769] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:32:18 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:32:19.209172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7a7ac5a0-abca-4567-9a18-1f0fea986395 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:19.209731 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7a7ac5a0-abca-4567-9a18-1f0fea986395 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:19.216276 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7a7ac5a0-abca-4567-9a18-1f0fea986395 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:19.216760 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7a7ac5a0-abca-4567-9a18-1f0fea986395 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:19.217196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7a7ac5a0-abca-4567-9a18-1f0fea986395 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:19.217883 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7a7ac5a0-abca-4567-9a18-1f0fea986395 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:19.218387 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7a7ac5a0-abca-4567-9a18-1f0fea986395 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:19.218815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7a7ac5a0-abca-4567-9a18-1f0fea986395 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:19.349671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7a7ac5a0-abca-4567-9a18-1f0fea986395 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:19.358717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7a7ac5a0-abca-4567-9a18-1f0fea986395 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:19.360158 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7a7ac5a0-abca-4567-9a18-1f0fea986395 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:19.361330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7a7ac5a0-abca-4567-9a18-1f0fea986395 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1726 microversion: 2.20 time: 0.156250 Sep 08 14:32:19.362853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1527/2770] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:32:19 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1726 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:32:20.387169 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-15c7e30c-5ecc-4dd4-8d0a-e7f4fe5adfb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:20.387890 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-15c7e30c-5ecc-4dd4-8d0a-e7f4fe5adfb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:20.394934 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-15c7e30c-5ecc-4dd4-8d0a-e7f4fe5adfb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:20.395427 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-15c7e30c-5ecc-4dd4-8d0a-e7f4fe5adfb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:20.397304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-15c7e30c-5ecc-4dd4-8d0a-e7f4fe5adfb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:20.397304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-15c7e30c-5ecc-4dd4-8d0a-e7f4fe5adfb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:20.397304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-15c7e30c-5ecc-4dd4-8d0a-e7f4fe5adfb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:20.397773 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-15c7e30c-5ecc-4dd4-8d0a-e7f4fe5adfb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:20.530601 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-15c7e30c-5ecc-4dd4-8d0a-e7f4fe5adfb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:20.538775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-15c7e30c-5ecc-4dd4-8d0a-e7f4fe5adfb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:20.540253 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-15c7e30c-5ecc-4dd4-8d0a-e7f4fe5adfb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:20.541418 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-15c7e30c-5ecc-4dd4-8d0a-e7f4fe5adfb9 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1713 microversion: 2.20 time: 0.158161 Sep 08 14:32:20.541982 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1244/2771] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:32:20 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1713 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:32:58.231935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-33db6b5c-bed1-4b7f-ac84-94d40d71f662 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:58.232633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-33db6b5c-bed1-4b7f-ac84-94d40d71f662 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:58.235455 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=31 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:58.240169 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=84,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:58.240830 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-33db6b5c-bed1-4b7f-ac84-94d40d71f662 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:58.241309 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-33db6b5c-bed1-4b7f-ac84-94d40d71f662 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:58.241735 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-33db6b5c-bed1-4b7f-ac84-94d40d71f662 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:58.242451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-33db6b5c-bed1-4b7f-ac84-94d40d71f662 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:58.242899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-33db6b5c-bed1-4b7f-ac84-94d40d71f662 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:58.243362 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-33db6b5c-bed1-4b7f-ac84-94d40d71f662 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:58.525434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-33db6b5c-bed1-4b7f-ac84-94d40d71f662 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lazy-loading 'services' on Instance uuid c09b905a-0795-4dc5-a413-007f3951e49f {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:32:58.696777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-33db6b5c-bed1-4b7f-ac84-94d40d71f662 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f/os-volume_attachments/4fc84b59-b08f-4b94-839c-0dbb61139175" status: 202 len: 0 microversion: 2.1 time: 0.469084 Sep 08 14:32:58.697473 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1528/2772] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:32:58 2022] DELETE /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f/os-volume_attachments/4fc84b59-b08f-4b94-839c-0dbb61139175 => generated 0 bytes in 471 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:32:59.102605 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7ead0171-ab06-445c-a0b8-9de230bdc117 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:59.104841 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7ead0171-ab06-445c-a0b8-9de230bdc117 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:59.107001 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=7 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:59.107668 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=55 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:32:59.112813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7ead0171-ab06-445c-a0b8-9de230bdc117 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:59.113288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7ead0171-ab06-445c-a0b8-9de230bdc117 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:59.113725 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7ead0171-ab06-445c-a0b8-9de230bdc117 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:59.114446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7ead0171-ab06-445c-a0b8-9de230bdc117 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:59.117474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7ead0171-ab06-445c-a0b8-9de230bdc117 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:59.117474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7ead0171-ab06-445c-a0b8-9de230bdc117 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:59.147141 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7ead0171-ab06-445c-a0b8-9de230bdc117 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.048571 Sep 08 14:32:59.147691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1245/2773] 10.222.0.25 () {60 vars in 1343 bytes} [Thu Sep 8 14:32:59 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9/os-volume_attachments => generated 25 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:32:59.215081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-22fe10d8-6f76-46d5-9b45-0d95aa21fdb6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:59.215776 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-22fe10d8-6f76-46d5-9b45-0d95aa21fdb6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:59.222356 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22fe10d8-6f76-46d5-9b45-0d95aa21fdb6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:59.222853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22fe10d8-6f76-46d5-9b45-0d95aa21fdb6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:59.223294 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22fe10d8-6f76-46d5-9b45-0d95aa21fdb6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:59.223993 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22fe10d8-6f76-46d5-9b45-0d95aa21fdb6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:59.224452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22fe10d8-6f76-46d5-9b45-0d95aa21fdb6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:59.224906 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-22fe10d8-6f76-46d5-9b45-0d95aa21fdb6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:59.251876 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-22fe10d8-6f76-46d5-9b45-0d95aa21fdb6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:32:59.600491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-22fe10d8-6f76-46d5-9b45-0d95aa21fdb6 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 204 len: 0 microversion: 2.20 time: 0.391428 Sep 08 14:32:59.601082 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1529/2774] 10.222.0.25 () {60 vars in 1280 bytes} [Thu Sep 8 14:32:59 2022] DELETE /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 08 14:32:59.611752 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4d3da5fa-428e-4137-aa7a-1954874c0f99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:32:59.612362 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4d3da5fa-428e-4137-aa7a-1954874c0f99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:32:59.619048 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d3da5fa-428e-4137-aa7a-1954874c0f99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:59.619524 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d3da5fa-428e-4137-aa7a-1954874c0f99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:59.619957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d3da5fa-428e-4137-aa7a-1954874c0f99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:59.620696 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d3da5fa-428e-4137-aa7a-1954874c0f99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:32:59.621150 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d3da5fa-428e-4137-aa7a-1954874c0f99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:32:59.621568 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4d3da5fa-428e-4137-aa7a-1954874c0f99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:32:59.856973 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4d3da5fa-428e-4137-aa7a-1954874c0f99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:59.865894 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4d3da5fa-428e-4137-aa7a-1954874c0f99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:59.867388 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4d3da5fa-428e-4137-aa7a-1954874c0f99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:32:59.868541 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4d3da5fa-428e-4137-aa7a-1954874c0f99 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.261673 Sep 08 14:32:59.869096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1246/2775] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:32:59 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:33:00.880193 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d27e8391-b65e-42ab-acbe-b0b0778ac4b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:33:00.886630 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d27e8391-b65e-42ab-acbe-b0b0778ac4b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:00.887114 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d27e8391-b65e-42ab-acbe-b0b0778ac4b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:00.887542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d27e8391-b65e-42ab-acbe-b0b0778ac4b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:00.894106 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-beed7772-5d64-4ca0-8fbd-4bc0c4a7ba11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:00.894741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-beed7772-5d64-4ca0-8fbd-4bc0c4a7ba11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:00.901591 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-beed7772-5d64-4ca0-8fbd-4bc0c4a7ba11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:00.902077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-beed7772-5d64-4ca0-8fbd-4bc0c4a7ba11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:00.902553 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-beed7772-5d64-4ca0-8fbd-4bc0c4a7ba11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:00.903248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-beed7772-5d64-4ca0-8fbd-4bc0c4a7ba11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:00.903703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-beed7772-5d64-4ca0-8fbd-4bc0c4a7ba11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:00.904126 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-beed7772-5d64-4ca0-8fbd-4bc0c4a7ba11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:00.909869 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d27e8391-b65e-42ab-acbe-b0b0778ac4b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Going to try to stop instance {{(pid=127601) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 08 14:33:01.053365 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-beed7772-5d64-4ca0-8fbd-4bc0c4a7ba11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:01.062504 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-beed7772-5d64-4ca0-8fbd-4bc0c4a7ba11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:01.064366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-beed7772-5d64-4ca0-8fbd-4bc0c4a7ba11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:01.065559 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-beed7772-5d64-4ca0-8fbd-4bc0c4a7ba11 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.175648 Sep 08 14:33:01.066124 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1247/2776] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:33:00 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 08 14:33:01.141417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d27e8391-b65e-42ab-acbe-b0b0778ac4b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "POST /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f/action" status: 202 len: 0 microversion: 2.1 time: 0.265823 Sep 08 14:33:01.142987 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1530/2777] 10.222.0.25 () {60 vars in 1278 bytes} [Thu Sep 8 14:33:00 2022] POST /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f/action => generated 0 bytes in 269 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:33:01.151307 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c176886a-b20e-4923-8d64-9528d1b89ba3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:01.151917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c176886a-b20e-4923-8d64-9528d1b89ba3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:01.157957 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c176886a-b20e-4923-8d64-9528d1b89ba3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:01.158467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c176886a-b20e-4923-8d64-9528d1b89ba3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:01.158903 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c176886a-b20e-4923-8d64-9528d1b89ba3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:01.159617 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c176886a-b20e-4923-8d64-9528d1b89ba3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:01.160037 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c176886a-b20e-4923-8d64-9528d1b89ba3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:01.160458 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c176886a-b20e-4923-8d64-9528d1b89ba3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:01.315443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c176886a-b20e-4923-8d64-9528d1b89ba3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:01.324083 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c176886a-b20e-4923-8d64-9528d1b89ba3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1676 microversion: 2.1 time: 0.176622 Sep 08 14:33:01.324668 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1248/2778] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:01 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1676 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:33:02.093054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-63b41359-09a6-402c-a39e-41ae4c184f2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:02.093664 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-63b41359-09a6-402c-a39e-41ae4c184f2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:02.100451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-63b41359-09a6-402c-a39e-41ae4c184f2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:02.100941 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-63b41359-09a6-402c-a39e-41ae4c184f2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:02.101381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-63b41359-09a6-402c-a39e-41ae4c184f2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:02.102079 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-63b41359-09a6-402c-a39e-41ae4c184f2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:02.102563 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-63b41359-09a6-402c-a39e-41ae4c184f2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:02.102989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-63b41359-09a6-402c-a39e-41ae4c184f2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:02.349444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-63b41359-09a6-402c-a39e-41ae4c184f2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:02.349986 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-008f5b19-bf56-4cd1-b80d-554f70ba92c3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:02.351886 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-008f5b19-bf56-4cd1-b80d-554f70ba92c3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:02.358839 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-63b41359-09a6-402c-a39e-41ae4c184f2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:02.360418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-63b41359-09a6-402c-a39e-41ae4c184f2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'e9925313d1d446fa81a00cb55348dff7', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '4c194dd986d442d3bcaa83442a49ff70', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:02.361947 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-63b41359-09a6-402c-a39e-41ae4c184f2c tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 200 len: 1719 microversion: 2.20 time: 0.274079 Sep 08 14:33:02.362454 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-008f5b19-bf56-4cd1-b80d-554f70ba92c3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:02.363175 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-008f5b19-bf56-4cd1-b80d-554f70ba92c3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:02.363813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-008f5b19-bf56-4cd1-b80d-554f70ba92c3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:02.364626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-008f5b19-bf56-4cd1-b80d-554f70ba92c3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:02.365264 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-008f5b19-bf56-4cd1-b80d-554f70ba92c3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:02.365783 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-008f5b19-bf56-4cd1-b80d-554f70ba92c3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:02.374782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1531/2779] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:33:02 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 1719 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 08 14:33:02.507882 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-008f5b19-bf56-4cd1-b80d-554f70ba92c3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:02.515820 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-008f5b19-bf56-4cd1-b80d-554f70ba92c3 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1676 microversion: 2.1 time: 0.171191 Sep 08 14:33:02.516401 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1249/2780] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:02 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1676 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:33:02.613527 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-ae7ec618-93f4-467d-b510-2ad65873f42d req-abdbe52d-6659-4959-a522-2fef1bf24f2f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "b0741665-d24b-4c17-93c7-90bcfff633b9", "name": "network-vif-deleted", "tag": "67fe80b7-8ae7-46be-8809-bd4958623c19"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:33:02.627419 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-ae7ec618-93f4-467d-b510-2ad65873f42d req-abdbe52d-6659-4959-a522-2fef1bf24f2f service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:02.627905 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-ae7ec618-93f4-467d-b510-2ad65873f42d req-abdbe52d-6659-4959-a522-2fef1bf24f2f service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:02.628536 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-ae7ec618-93f4-467d-b510-2ad65873f42d req-abdbe52d-6659-4959-a522-2fef1bf24f2f service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:02.645321 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-ae7ec618-93f4-467d-b510-2ad65873f42d req-abdbe52d-6659-4959-a522-2fef1bf24f2f service nova] Creating event network-vif-deleted:67fe80b7-8ae7-46be-8809-bd4958623c19 for instance b0741665-d24b-4c17-93c7-90bcfff633b9 on hv-cinder-85646 Sep 08 14:33:02.651361 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-ae7ec618-93f4-467d-b510-2ad65873f42d req-abdbe52d-6659-4959-a522-2fef1bf24f2f service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041493 Sep 08 14:33:02.656394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1532/2781] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:33:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:33:03.389686 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d81df24f-c6c0-43a0-9165-ce9e68745a67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:03.390431 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d81df24f-c6c0-43a0-9165-ce9e68745a67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:03.397621 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d81df24f-c6c0-43a0-9165-ce9e68745a67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:03.398115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d81df24f-c6c0-43a0-9165-ce9e68745a67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:03.398583 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d81df24f-c6c0-43a0-9165-ce9e68745a67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:03.399288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d81df24f-c6c0-43a0-9165-ce9e68745a67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:03.399738 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d81df24f-c6c0-43a0-9165-ce9e68745a67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:03.400176 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d81df24f-c6c0-43a0-9165-ce9e68745a67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:03.415540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-d81df24f-c6c0-43a0-9165-ce9e68745a67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] HTTP exception thrown: Instance b0741665-d24b-4c17-93c7-90bcfff633b9 could not be found. Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d81df24f-c6c0-43a0-9165-ce9e68745a67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Request method failure captured: Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 HTTP/1.1 Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPpgseYqW3vfBhChnnflETRt0GDtjL3VmhITeNLuRSALZefRpZ4LlXgFeo5cygwUtHgGuuBrrkR39Uo-ZWGIlwSJVssX0SoJbjjdLhYxjZOutCiSryd_0UQo0F3jz05Y7DAMd5jNO7kUKOE8WKEghkWPfZegSJZuLFAO1D7uF_weRo Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Openstack-Nova-Api-Version: 2.20 Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:33:03.417220 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:33:03.419246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:33:03.419246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:33:03.419246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:33:03.419246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:33:03.419246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:33:03.419246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:33:03.419246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:33:03.419246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: e9925313d1d446fa81a00cb55348dff7 Sep 08 14:33:03.419246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:33:03.419246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:33:03.419246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance b0741665-d24b-4c17-93c7-90bcfff633b9 could not be found. Sep 08 14:33:03.419246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'b0741665-d24b-4c17-93c7-90bcfff633b9'} Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b0741665-d24b-4c17-93c7-90bcfff633b9 could not be found. Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:33:03.420467 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:33:03.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:33:03.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:33:03.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:33:03.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:33:03.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:33:03.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:33:03.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:33:03.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b0741665-d24b-4c17-93c7-90bcfff633b9 could not be found. Sep 08 14:33:03.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:33:03.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:33:03.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:33:03.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:33:03.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:33:03.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:33:03.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:33:03.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:33:03.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b0741665-d24b-4c17-93c7-90bcfff633b9 could not be found. Sep 08 14:33:03.422252 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:33:03.424906 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d81df24f-c6c0-43a0-9165-ce9e68745a67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Returning 404 to user: Instance b0741665-d24b-4c17-93c7-90bcfff633b9 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:33:03.424906 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d81df24f-c6c0-43a0-9165-ce9e68745a67 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 404 len: 111 microversion: 2.20 time: 0.032931 Sep 08 14:33:03.424906 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1250/2782] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:33:03 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 08 14:33:03.436100 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-12e91856-7a81-4f8e-a2c4-f86199264408 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '.version_select at 0x7fce01ab4160>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:03.540287 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7719c8ff-f887-4337-8451-24082750fa2a tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:03.540907 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7719c8ff-f887-4337-8451-24082750fa2a tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:03.547226 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7719c8ff-f887-4337-8451-24082750fa2a tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:03.547710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7719c8ff-f887-4337-8451-24082750fa2a tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:03.548719 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7719c8ff-f887-4337-8451-24082750fa2a tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:03.549697 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7719c8ff-f887-4337-8451-24082750fa2a tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:03.550178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7719c8ff-f887-4337-8451-24082750fa2a tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:03.550607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7719c8ff-f887-4337-8451-24082750fa2a tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:03.550866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-12e91856-7a81-4f8e-a2c4-f86199264408 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-884379298" status: 204 len: 0 microversion: 2.20 time: 0.124030 Sep 08 14:33:03.551745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1533/2783] 10.222.0.25 () {60 vars in 1259 bytes} [Thu Sep 8 14:33:03 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-884379298 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 08 14:33:03.752178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7719c8ff-f887-4337-8451-24082750fa2a tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:03.760366 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7719c8ff-f887-4337-8451-24082750fa2a tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1676 microversion: 2.1 time: 0.223773 Sep 08 14:33:03.760950 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1251/2784] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:03 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1676 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:33:04.785865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5f673644-b836-4c83-b81c-f3e8a831be8e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:04.786594 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5f673644-b836-4c83-b81c-f3e8a831be8e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:04.793474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f673644-b836-4c83-b81c-f3e8a831be8e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:04.793964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f673644-b836-4c83-b81c-f3e8a831be8e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:04.794442 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f673644-b836-4c83-b81c-f3e8a831be8e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:04.795132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f673644-b836-4c83-b81c-f3e8a831be8e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:04.795586 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f673644-b836-4c83-b81c-f3e8a831be8e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:04.796012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5f673644-b836-4c83-b81c-f3e8a831be8e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:04.941818 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5f673644-b836-4c83-b81c-f3e8a831be8e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:04.952342 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5f673644-b836-4c83-b81c-f3e8a831be8e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1676 microversion: 2.1 time: 0.170782 Sep 08 14:33:04.952921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1534/2785] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:04 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1676 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:33:05.730978 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0ec89cde-9171-4004-aa30-64883cfe7ec7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:05.731771 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-0ec89cde-9171-4004-aa30-64883cfe7ec7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:05.738396 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ec89cde-9171-4004-aa30-64883cfe7ec7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:05.738879 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ec89cde-9171-4004-aa30-64883cfe7ec7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:05.739308 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ec89cde-9171-4004-aa30-64883cfe7ec7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:05.739998 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ec89cde-9171-4004-aa30-64883cfe7ec7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:05.740440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ec89cde-9171-4004-aa30-64883cfe7ec7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:05.740855 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-0ec89cde-9171-4004-aa30-64883cfe7ec7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:05.755808 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-0ec89cde-9171-4004-aa30-64883cfe7ec7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] HTTP exception thrown: Instance b0741665-d24b-4c17-93c7-90bcfff633b9 could not be found. Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0ec89cde-9171-4004-aa30-64883cfe7ec7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Request method failure captured: Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 HTTP/1.1 Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPpgseYqW3vfBhChnnflETRt0GDtjL3VmhITeNLuRSALZefRpZ4LlXgFeo5cygwUtHgGuuBrrkR39Uo-ZWGIlwSJVssX0SoJbjjdLhYxjZOutCiSryd_0UQo0F3jz05Y7DAMd5jNO7kUKOE8WKEghkWPfZegSJZuLFAO1D7uF_weRo Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Openstack-Nova-Api-Version: 2.20 Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:33:05.757373 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:33:05.760775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:33:05.760775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:33:05.760775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:33:05.760775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:33:05.760775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:33:05.760775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:33:05.760775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:33:05.760775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: e9925313d1d446fa81a00cb55348dff7 Sep 08 14:33:05.760775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:33:05.760775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:33:05.760775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance b0741665-d24b-4c17-93c7-90bcfff633b9 could not be found. Sep 08 14:33:05.760775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'b0741665-d24b-4c17-93c7-90bcfff633b9'} Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b0741665-d24b-4c17-93c7-90bcfff633b9 could not be found. Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:33:05.762729 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b0741665-d24b-4c17-93c7-90bcfff633b9 could not be found. Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b0741665-d24b-4c17-93c7-90bcfff633b9 could not be found. Sep 08 14:33:05.764540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:33:05.766367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-0ec89cde-9171-4004-aa30-64883cfe7ec7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Returning 404 to user: Instance b0741665-d24b-4c17-93c7-90bcfff633b9 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:33:05.766367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-0ec89cde-9171-4004-aa30-64883cfe7ec7 tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 404 len: 111 microversion: 2.20 time: 0.032069 Sep 08 14:33:05.766367 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1252/2786] 10.222.0.25 () {60 vars in 1280 bytes} [Thu Sep 8 14:33:05 2022] DELETE /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Sep 08 14:33:05.768866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d51530d8-3afa-4aa7-bc7f-2d42c9eba9ad tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:05.769471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d51530d8-3afa-4aa7-bc7f-2d42c9eba9ad tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: b0741665-d24b-4c17-93c7-90bcfff633b9] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:05.776063 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d51530d8-3afa-4aa7-bc7f-2d42c9eba9ad tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:05.776551 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d51530d8-3afa-4aa7-bc7f-2d42c9eba9ad tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:05.776990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d51530d8-3afa-4aa7-bc7f-2d42c9eba9ad tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:05.777691 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d51530d8-3afa-4aa7-bc7f-2d42c9eba9ad tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:05.778172 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d51530d8-3afa-4aa7-bc7f-2d42c9eba9ad tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:05.778599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d51530d8-3afa-4aa7-bc7f-2d42c9eba9ad tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:05.794791 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-d51530d8-3afa-4aa7-bc7f-2d42c9eba9ad tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] HTTP exception thrown: Instance b0741665-d24b-4c17-93c7-90bcfff633b9 could not be found. Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d51530d8-3afa-4aa7-bc7f-2d42c9eba9ad tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Request method failure captured: Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 HTTP/1.1 Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPpgseYqW3vfBhChnnflETRt0GDtjL3VmhITeNLuRSALZefRpZ4LlXgFeo5cygwUtHgGuuBrrkR39Uo-ZWGIlwSJVssX0SoJbjjdLhYxjZOutCiSryd_0UQo0F3jz05Y7DAMd5jNO7kUKOE8WKEghkWPfZegSJZuLFAO1D7uF_weRo Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Openstack-Nova-Api-Version: 2.20 Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:33:05.796675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:33:05.800742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:33:05.800742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:33:05.800742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:33:05.800742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:33:05.800742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:33:05.800742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:33:05.800742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:33:05.800742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: e9925313d1d446fa81a00cb55348dff7 Sep 08 14:33:05.800742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:33:05.800742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:33:05.800742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance b0741665-d24b-4c17-93c7-90bcfff633b9 could not be found. Sep 08 14:33:05.800742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'b0741665-d24b-4c17-93c7-90bcfff633b9'} Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance b0741665-d24b-4c17-93c7-90bcfff633b9 could not be found. Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:33:05.801860 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:33:05.803637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:33:05.803637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:33:05.803637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:33:05.803637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:33:05.803637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:33:05.803637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:33:05.803637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:33:05.803637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance b0741665-d24b-4c17-93c7-90bcfff633b9 could not be found. Sep 08 14:33:05.803637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:33:05.803637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:33:05.803637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:33:05.803637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:33:05.803637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:33:05.803637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:33:05.803637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:33:05.803637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:33:05.803637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance b0741665-d24b-4c17-93c7-90bcfff633b9 could not be found. Sep 08 14:33:05.803637 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:33:05.805287 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d51530d8-3afa-4aa7-bc7f-2d42c9eba9ad tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Returning 404 to user: Instance b0741665-d24b-4c17-93c7-90bcfff633b9 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:33:05.805287 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d51530d8-3afa-4aa7-bc7f-2d42c9eba9ad tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9" status: 404 len: 111 microversion: 2.20 time: 0.034423 Sep 08 14:33:05.805287 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1535/2787] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:33:05 2022] GET /compute/v2.1/servers/b0741665-d24b-4c17-93c7-90bcfff633b9 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 08 14:33:05.976097 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6c5f399e-b713-4c53-bc6a-b5bf79b0e9e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:05.976712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6c5f399e-b713-4c53-bc6a-b5bf79b0e9e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:05.982738 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c5f399e-b713-4c53-bc6a-b5bf79b0e9e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:05.983224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c5f399e-b713-4c53-bc6a-b5bf79b0e9e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:05.983667 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c5f399e-b713-4c53-bc6a-b5bf79b0e9e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:05.984358 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c5f399e-b713-4c53-bc6a-b5bf79b0e9e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:05.984818 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c5f399e-b713-4c53-bc6a-b5bf79b0e9e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:05.985246 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6c5f399e-b713-4c53-bc6a-b5bf79b0e9e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:06.124350 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6c5f399e-b713-4c53-bc6a-b5bf79b0e9e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:06.132140 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6c5f399e-b713-4c53-bc6a-b5bf79b0e9e2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1676 microversion: 2.1 time: 0.159985 Sep 08 14:33:06.132701 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1253/2788] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:05 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1676 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:33:07.155673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-68cb180b-daec-4cfc-9137-5a337f8f6752 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:07.156207 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-68cb180b-daec-4cfc-9137-5a337f8f6752 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:07.163422 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68cb180b-daec-4cfc-9137-5a337f8f6752 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:07.163910 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68cb180b-daec-4cfc-9137-5a337f8f6752 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:07.164349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68cb180b-daec-4cfc-9137-5a337f8f6752 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:07.165036 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68cb180b-daec-4cfc-9137-5a337f8f6752 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:07.165499 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68cb180b-daec-4cfc-9137-5a337f8f6752 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:07.165917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68cb180b-daec-4cfc-9137-5a337f8f6752 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:07.297984 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-68cb180b-daec-4cfc-9137-5a337f8f6752 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:07.312058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-68cb180b-daec-4cfc-9137-5a337f8f6752 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1676 microversion: 2.1 time: 0.157081 Sep 08 14:33:07.312058 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1536/2789] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:07 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1676 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:33:08.332945 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-9b5aa3b0-b0ec-435d-83ed-eca6720dbc1b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:08.333558 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-9b5aa3b0-b0ec-435d-83ed-eca6720dbc1b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:08.339781 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9b5aa3b0-b0ec-435d-83ed-eca6720dbc1b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:08.340268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9b5aa3b0-b0ec-435d-83ed-eca6720dbc1b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:08.340699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9b5aa3b0-b0ec-435d-83ed-eca6720dbc1b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:08.341389 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9b5aa3b0-b0ec-435d-83ed-eca6720dbc1b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:08.341836 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9b5aa3b0-b0ec-435d-83ed-eca6720dbc1b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:08.342282 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-9b5aa3b0-b0ec-435d-83ed-eca6720dbc1b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:08.479434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-9b5aa3b0-b0ec-435d-83ed-eca6720dbc1b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:08.489826 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-9b5aa3b0-b0ec-435d-83ed-eca6720dbc1b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1676 microversion: 2.1 time: 0.159152 Sep 08 14:33:08.489826 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1254/2790] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:08 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1676 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:33:09.511883 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-df1ed6cf-4590-4062-aa1b-08842064c941 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:09.512530 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-df1ed6cf-4590-4062-aa1b-08842064c941 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:09.519513 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df1ed6cf-4590-4062-aa1b-08842064c941 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:09.520010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df1ed6cf-4590-4062-aa1b-08842064c941 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:09.520451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df1ed6cf-4590-4062-aa1b-08842064c941 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:09.521157 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df1ed6cf-4590-4062-aa1b-08842064c941 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:09.521614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df1ed6cf-4590-4062-aa1b-08842064c941 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:09.522049 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-df1ed6cf-4590-4062-aa1b-08842064c941 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:09.656037 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-df1ed6cf-4590-4062-aa1b-08842064c941 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:09.664295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-df1ed6cf-4590-4062-aa1b-08842064c941 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1676 microversion: 2.1 time: 0.156421 Sep 08 14:33:09.664863 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1537/2791] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:09 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1676 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:33:10.688249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a8814a86-ead4-4272-9a78-38750bef6b4b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:10.688880 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a8814a86-ead4-4272-9a78-38750bef6b4b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:10.696072 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a8814a86-ead4-4272-9a78-38750bef6b4b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:10.696551 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a8814a86-ead4-4272-9a78-38750bef6b4b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:10.696977 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a8814a86-ead4-4272-9a78-38750bef6b4b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:10.697663 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a8814a86-ead4-4272-9a78-38750bef6b4b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:10.698170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a8814a86-ead4-4272-9a78-38750bef6b4b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:10.698627 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a8814a86-ead4-4272-9a78-38750bef6b4b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:10.831028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a8814a86-ead4-4272-9a78-38750bef6b4b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:10.838858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a8814a86-ead4-4272-9a78-38750bef6b4b tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1676 microversion: 2.1 time: 0.154460 Sep 08 14:33:10.839426 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1255/2792] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:10 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1676 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:33:11.862489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c7a4c7b9-fb47-4de2-9a26-40ac45c19bae tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:11.863179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c7a4c7b9-fb47-4de2-9a26-40ac45c19bae tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:11.869973 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c7a4c7b9-fb47-4de2-9a26-40ac45c19bae tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:11.870505 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c7a4c7b9-fb47-4de2-9a26-40ac45c19bae tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:11.870982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c7a4c7b9-fb47-4de2-9a26-40ac45c19bae tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:11.871758 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c7a4c7b9-fb47-4de2-9a26-40ac45c19bae tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:11.872201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c7a4c7b9-fb47-4de2-9a26-40ac45c19bae tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:11.872626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c7a4c7b9-fb47-4de2-9a26-40ac45c19bae tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:12.009970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c7a4c7b9-fb47-4de2-9a26-40ac45c19bae tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:12.018881 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c7a4c7b9-fb47-4de2-9a26-40ac45c19bae tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1676 microversion: 2.1 time: 0.160421 Sep 08 14:33:12.019451 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1538/2793] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:11 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1676 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:33:13.042684 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ff8f81b2-fd35-463a-9881-d7a1de091aa4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:13.043292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ff8f81b2-fd35-463a-9881-d7a1de091aa4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:13.049876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff8f81b2-fd35-463a-9881-d7a1de091aa4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:13.050393 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff8f81b2-fd35-463a-9881-d7a1de091aa4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:13.050834 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff8f81b2-fd35-463a-9881-d7a1de091aa4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:13.051533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff8f81b2-fd35-463a-9881-d7a1de091aa4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:13.051987 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff8f81b2-fd35-463a-9881-d7a1de091aa4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:13.052414 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ff8f81b2-fd35-463a-9881-d7a1de091aa4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:13.190661 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ff8f81b2-fd35-463a-9881-d7a1de091aa4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:13.198789 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ff8f81b2-fd35-463a-9881-d7a1de091aa4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1653 microversion: 2.1 time: 0.159846 Sep 08 14:33:13.199302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1256/2794] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:13 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1653 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:33:13.222349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-51823bb5-a9dc-4349-9637-0b9de10b0d95 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:33:13.229066 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51823bb5-a9dc-4349-9637-0b9de10b0d95 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:13.229566 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51823bb5-a9dc-4349-9637-0b9de10b0d95 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:13.230012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-51823bb5-a9dc-4349-9637-0b9de10b0d95 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:13.257083 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-51823bb5-a9dc-4349-9637-0b9de10b0d95 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Going to try to start instance {{(pid=127601) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 08 14:33:13.427185 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-51823bb5-a9dc-4349-9637-0b9de10b0d95 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "POST /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f/action" status: 202 len: 0 microversion: 2.1 time: 0.209206 Sep 08 14:33:13.429658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1539/2795] 10.222.0.25 () {60 vars in 1278 bytes} [Thu Sep 8 14:33:13 2022] POST /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f/action => generated 0 bytes in 211 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:33:13.436179 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6bdb95c0-9891-4bc3-9558-521ad5984a44 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:13.436810 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6bdb95c0-9891-4bc3-9558-521ad5984a44 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:13.444067 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6bdb95c0-9891-4bc3-9558-521ad5984a44 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:13.444572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6bdb95c0-9891-4bc3-9558-521ad5984a44 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:13.445202 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6bdb95c0-9891-4bc3-9558-521ad5984a44 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:13.445712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6bdb95c0-9891-4bc3-9558-521ad5984a44 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:13.446191 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6bdb95c0-9891-4bc3-9558-521ad5984a44 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:13.446624 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6bdb95c0-9891-4bc3-9558-521ad5984a44 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:13.581558 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6bdb95c0-9891-4bc3-9558-521ad5984a44 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:13.589415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6bdb95c0-9891-4bc3-9558-521ad5984a44 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1662 microversion: 2.1 time: 0.157326 Sep 08 14:33:13.590002 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1257/2796] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:13 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1662 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:33:14.613598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-32af2275-d0a4-4248-880a-4b60ca6557b0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:14.614414 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-32af2275-d0a4-4248-880a-4b60ca6557b0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:14.620614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32af2275-d0a4-4248-880a-4b60ca6557b0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:14.621096 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32af2275-d0a4-4248-880a-4b60ca6557b0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:14.621569 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32af2275-d0a4-4248-880a-4b60ca6557b0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:14.622304 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32af2275-d0a4-4248-880a-4b60ca6557b0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:14.622769 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32af2275-d0a4-4248-880a-4b60ca6557b0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:14.623198 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-32af2275-d0a4-4248-880a-4b60ca6557b0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:14.753922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-32af2275-d0a4-4248-880a-4b60ca6557b0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:14.761883 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-32af2275-d0a4-4248-880a-4b60ca6557b0 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1662 microversion: 2.1 time: 0.153084 Sep 08 14:33:14.762483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1540/2797] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:14 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1662 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:33:15.788794 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-142db705-ad5d-4b43-833d-3a92ed40cf35 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:15.789364 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-142db705-ad5d-4b43-833d-3a92ed40cf35 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:15.795980 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-142db705-ad5d-4b43-833d-3a92ed40cf35 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:15.796465 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-142db705-ad5d-4b43-833d-3a92ed40cf35 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:15.796897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-142db705-ad5d-4b43-833d-3a92ed40cf35 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:15.797588 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-142db705-ad5d-4b43-833d-3a92ed40cf35 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:15.798042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-142db705-ad5d-4b43-833d-3a92ed40cf35 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:15.798495 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-142db705-ad5d-4b43-833d-3a92ed40cf35 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:15.934825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-142db705-ad5d-4b43-833d-3a92ed40cf35 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:15.942548 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-142db705-ad5d-4b43-833d-3a92ed40cf35 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1666 microversion: 2.1 time: 0.157733 Sep 08 14:33:15.943121 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1258/2798] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:15 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1666 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:33:51.404012 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5cc19231-2d90-4667-841b-e8eecf92fbb5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:51.406438 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5cc19231-2d90-4667-841b-e8eecf92fbb5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:51.408688 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=32,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:33:51.409388 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=83,nova_cell1:INSERT=4,nova_cell1:UPDATE=3 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:33:51.415127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5cc19231-2d90-4667-841b-e8eecf92fbb5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:51.415627 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5cc19231-2d90-4667-841b-e8eecf92fbb5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:51.416064 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5cc19231-2d90-4667-841b-e8eecf92fbb5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:51.416866 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5cc19231-2d90-4667-841b-e8eecf92fbb5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:51.417323 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5cc19231-2d90-4667-841b-e8eecf92fbb5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:51.417748 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5cc19231-2d90-4667-841b-e8eecf92fbb5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:51.452315 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5cc19231-2d90-4667-841b-e8eecf92fbb5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.052084 Sep 08 14:33:51.452625 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1541/2799] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:33:51 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f/os-volume_attachments => generated 25 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 08 14:33:51.519446 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-33f86aa4-9fed-424a-8eab-6cc2a237a8ce tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:51.520028 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-33f86aa4-9fed-424a-8eab-6cc2a237a8ce tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:51.521945 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=28 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:33:51.522791 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=76 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:33:51.523625 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:33:51.523769 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:33:51 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f (ip 10.222.0.25) !!! Sep 08 14:33:51.529287 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33f86aa4-9fed-424a-8eab-6cc2a237a8ce tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:51.529763 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33f86aa4-9fed-424a-8eab-6cc2a237a8ce tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:51.530215 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33f86aa4-9fed-424a-8eab-6cc2a237a8ce tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:51.530930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33f86aa4-9fed-424a-8eab-6cc2a237a8ce tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:51.531378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33f86aa4-9fed-424a-8eab-6cc2a237a8ce tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:51.531797 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-33f86aa4-9fed-424a-8eab-6cc2a237a8ce tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:51.559774 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-33f86aa4-9fed-424a-8eab-6cc2a237a8ce tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:33:51.818861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-33f86aa4-9fed-424a-8eab-6cc2a237a8ce tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 204 len: 0 microversion: 2.1 time: 0.302701 Sep 08 14:33:51.819955 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1259/2800] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:33:51 2022] DELETE /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:33:51.828340 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-c245579c-be54-4193-aa38-38bab23ac675 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:51.828988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-c245579c-be54-4193-aa38-38bab23ac675 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:51.837115 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c245579c-be54-4193-aa38-38bab23ac675 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:51.837610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c245579c-be54-4193-aa38-38bab23ac675 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:51.838065 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c245579c-be54-4193-aa38-38bab23ac675 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:51.838806 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c245579c-be54-4193-aa38-38bab23ac675 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:51.839278 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c245579c-be54-4193-aa38-38bab23ac675 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:51.839713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-c245579c-be54-4193-aa38-38bab23ac675 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:51.977648 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-c245579c-be54-4193-aa38-38bab23ac675 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:51.985538 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-c245579c-be54-4193-aa38-38bab23ac675 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1672 microversion: 2.1 time: 0.161099 Sep 08 14:33:51.986106 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1542/2801] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:51 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1672 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:33:53.018225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7db5bc7d-b998-4e8d-bd6b-8b9b25dac7d8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:53.018940 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7db5bc7d-b998-4e8d-bd6b-8b9b25dac7d8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:53.025850 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7db5bc7d-b998-4e8d-bd6b-8b9b25dac7d8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:53.026403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7db5bc7d-b998-4e8d-bd6b-8b9b25dac7d8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:53.026839 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7db5bc7d-b998-4e8d-bd6b-8b9b25dac7d8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:53.027529 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7db5bc7d-b998-4e8d-bd6b-8b9b25dac7d8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:53.027982 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7db5bc7d-b998-4e8d-bd6b-8b9b25dac7d8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:53.028406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7db5bc7d-b998-4e8d-bd6b-8b9b25dac7d8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:53.159266 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7db5bc7d-b998-4e8d-bd6b-8b9b25dac7d8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:53.167352 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7db5bc7d-b998-4e8d-bd6b-8b9b25dac7d8 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1672 microversion: 2.1 time: 0.153646 Sep 08 14:33:53.167935 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1260/2802] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:53 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1672 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:33:54.190719 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-81a8a7c7-e537-45be-832b-6b0509890a78 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:54.191363 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-81a8a7c7-e537-45be-832b-6b0509890a78 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:54.198704 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81a8a7c7-e537-45be-832b-6b0509890a78 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:54.199273 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81a8a7c7-e537-45be-832b-6b0509890a78 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:54.199745 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81a8a7c7-e537-45be-832b-6b0509890a78 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:54.200447 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81a8a7c7-e537-45be-832b-6b0509890a78 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:54.200907 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81a8a7c7-e537-45be-832b-6b0509890a78 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:54.201331 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81a8a7c7-e537-45be-832b-6b0509890a78 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:54.462795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-81a8a7c7-e537-45be-832b-6b0509890a78 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:54.462795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-81a8a7c7-e537-45be-832b-6b0509890a78 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1672 microversion: 2.1 time: 0.152270 Sep 08 14:33:54.462795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1543/2803] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:54 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1672 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:33:55.363725 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d169fcc6-de7c-4be8-a6aa-3b08fc3534b1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:55.364821 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d169fcc6-de7c-4be8-a6aa-3b08fc3534b1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:55.371415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d169fcc6-de7c-4be8-a6aa-3b08fc3534b1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:55.371888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d169fcc6-de7c-4be8-a6aa-3b08fc3534b1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:55.372314 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d169fcc6-de7c-4be8-a6aa-3b08fc3534b1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:55.372995 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d169fcc6-de7c-4be8-a6aa-3b08fc3534b1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:55.373443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d169fcc6-de7c-4be8-a6aa-3b08fc3534b1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:55.373853 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d169fcc6-de7c-4be8-a6aa-3b08fc3534b1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:55.445104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d169fcc6-de7c-4be8-a6aa-3b08fc3534b1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:55.454958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d169fcc6-de7c-4be8-a6aa-3b08fc3534b1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1605 microversion: 2.1 time: 0.094829 Sep 08 14:33:55.462317 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1261/2804] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:55 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1605 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:33:55.475410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-a178bc7d-0dca-42ba-a648-fe995e306a2f req-efa83890-b101-4f41-9df5-443eb50a5e52 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c09b905a-0795-4dc5-a413-007f3951e49f", "name": "network-vif-deleted", "tag": "c06eb27d-e611-488c-80be-0549fc42ecef"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:33:55.485514 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-a178bc7d-0dca-42ba-a648-fe995e306a2f req-efa83890-b101-4f41-9df5-443eb50a5e52 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:55.486166 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-a178bc7d-0dca-42ba-a648-fe995e306a2f req-efa83890-b101-4f41-9df5-443eb50a5e52 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:55.486713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-a178bc7d-0dca-42ba-a648-fe995e306a2f req-efa83890-b101-4f41-9df5-443eb50a5e52 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:55.500338 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-a178bc7d-0dca-42ba-a648-fe995e306a2f req-efa83890-b101-4f41-9df5-443eb50a5e52 service nova] Creating event network-vif-deleted:c06eb27d-e611-488c-80be-0549fc42ecef for instance c09b905a-0795-4dc5-a413-007f3951e49f on hv-cinder-85646 Sep 08 14:33:55.506319 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-a178bc7d-0dca-42ba-a648-fe995e306a2f req-efa83890-b101-4f41-9df5-443eb50a5e52 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037932 Sep 08 14:33:55.506953 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1544/2805] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:33:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:33:56.479843 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e6f2659b-7306-478d-a4bc-3e34d34565f2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:56.480375 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e6f2659b-7306-478d-a4bc-3e34d34565f2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:56.487009 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6f2659b-7306-478d-a4bc-3e34d34565f2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:56.487482 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6f2659b-7306-478d-a4bc-3e34d34565f2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:56.488007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6f2659b-7306-478d-a4bc-3e34d34565f2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:56.488689 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6f2659b-7306-478d-a4bc-3e34d34565f2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:56.489248 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6f2659b-7306-478d-a4bc-3e34d34565f2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:56.489679 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6f2659b-7306-478d-a4bc-3e34d34565f2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:56.523420 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6f2659b-7306-478d-a4bc-3e34d34565f2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:56.523896 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6f2659b-7306-478d-a4bc-3e34d34565f2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:56.524324 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e6f2659b-7306-478d-a4bc-3e34d34565f2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:56.524760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-e6f2659b-7306-478d-a4bc-3e34d34565f2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lazy-loading 'fault' on Instance uuid c09b905a-0795-4dc5-a413-007f3951e49f {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:33:56.573875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e6f2659b-7306-478d-a4bc-3e34d34565f2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:33:56.581992 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e6f2659b-7306-478d-a4bc-3e34d34565f2 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 200 len: 1326 microversion: 2.1 time: 0.105986 Sep 08 14:33:56.582582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1262/2806] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:56 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 1326 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:33:57.607854 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1a2f2365-4203-4020-8843-36bda9488757 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:57.608555 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1a2f2365-4203-4020-8843-36bda9488757 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:33:57.614892 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1a2f2365-4203-4020-8843-36bda9488757 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:57.615381 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1a2f2365-4203-4020-8843-36bda9488757 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:57.615817 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1a2f2365-4203-4020-8843-36bda9488757 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:57.616507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1a2f2365-4203-4020-8843-36bda9488757 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:33:57.616959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1a2f2365-4203-4020-8843-36bda9488757 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:33:57.617384 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1a2f2365-4203-4020-8843-36bda9488757 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:33:57.630908 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-1a2f2365-4203-4020-8843-36bda9488757 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] HTTP exception thrown: Instance c09b905a-0795-4dc5-a413-007f3951e49f could not be found. Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1a2f2365-4203-4020-8843-36bda9488757 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Request method failure captured: Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f HTTP/1.1 Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfx-IBlTeU8lpWwGKB_djClew3IfsnnMsqAlTIyJwZatK6rO97X1sIwHrl836QwkkBwsDAti13QWgcktV2H3C3H2iyUMQVmkZ2rvFbwK_qsxWVKARvFf1vGfBVG_NZnxLXL77ZYDqLGjePARIcvrYdLR_yhLC_R9BR-JLVq-P_eoaHc Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: b21f666643b74b5cba0b7597ae1aa0cd Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeTestJSON-790394194 Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:33:57.632528 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:33:57.633768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:33:57.633768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeTestJSON-790394194 Sep 08 14:33:57.633768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: b21f666643b74b5cba0b7597ae1aa0cd Sep 08 14:33:57.633768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeTestJSON-790394194 Sep 08 14:33:57.633768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeTestJSON-790394194-project Sep 08 14:33:57.633768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:33:57.633768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:33:57.633768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 357ebb86fd664e08b15bf440e2a61560 Sep 08 14:33:57.633768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeTestJSON-790394194-project Sep 08 14:33:57.633768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:33:57.633768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance c09b905a-0795-4dc5-a413-007f3951e49f could not be found. Sep 08 14:33:57.633768 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'c09b905a-0795-4dc5-a413-007f3951e49f'} Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c09b905a-0795-4dc5-a413-007f3951e49f could not be found. Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:33:57.634595 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:33:57.635497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:33:57.635497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:33:57.635497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:33:57.635497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:33:57.635497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:33:57.635497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:33:57.635497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:33:57.635497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c09b905a-0795-4dc5-a413-007f3951e49f could not be found. Sep 08 14:33:57.635497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:33:57.635497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:33:57.635497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:33:57.635497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:33:57.635497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:33:57.635497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:33:57.635497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:33:57.635497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:33:57.635497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c09b905a-0795-4dc5-a413-007f3951e49f could not be found. Sep 08 14:33:57.635497 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:33:57.636465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1a2f2365-4203-4020-8843-36bda9488757 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Returning 404 to user: Instance c09b905a-0795-4dc5-a413-007f3951e49f could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:33:57.636465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1a2f2365-4203-4020-8843-36bda9488757 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 404 len: 111 microversion: 2.1 time: 0.031040 Sep 08 14:33:57.636465 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1545/2807] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:33:57 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:33:57.642849 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e9ca63d2-832f-4f56-85c0-fc7483c6c6e1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '.version_select at 0x7fce0588faf0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:33:57.744733 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e9ca63d2-832f-4f56-85c0-fc7483c6c6e1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-338464191" status: 202 len: 0 microversion: 2.1 time: 0.105571 Sep 08 14:33:57.745301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1263/2808] 10.222.0.25 () {58 vars in 1218 bytes} [Thu Sep 8 14:33:57 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-338464191 => generated 0 bytes in 107 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:33:58.551561 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e936cc9b-8e9f-43ff-bebd-93525a1b1c4f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Action: 'create', calling method: .version_select at 0x7fce01b64f70>, body: {"keypair": {"name": "tempest-keypair-1741077166", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK5cvUJrhB8MqtlrAUA1tcw/2U7VPTGgOdrqKiEEt6PRUUA/bejE6gKZydKf+w5wZlDZ7EPXYzZB6tg2kDGgBrwuQd+FIbca/PqaVGVAFhqx+p1x08qaTsvRyAlHGDVyEQ=="}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:33:58.566716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-e936cc9b-8e9f-43ff-bebd-93525a1b1c4f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Getting quotas for project b21f666643b74b5cba0b7597ae1aa0cd. Resources: {'key_pairs'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 14:33:58.571201 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-e936cc9b-8e9f-43ff-bebd-93525a1b1c4f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Getting quotas for user 357ebb86fd664e08b15bf440e2a61560 and project b21f666643b74b5cba0b7597ae1aa0cd. Resources: {'key_pairs'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 14:33:58.647534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e936cc9b-8e9f-43ff-bebd-93525a1b1c4f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.099842 Sep 08 14:33:58.648137 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1546/2809] 10.222.0.25 () {60 vars in 1159 bytes} [Thu Sep 8 14:33:58 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:34:00.832115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-54383ccd-c832-4691-abf0-a1d56322e1fc tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-26212792", "imageRef": "20b30425-92e5-4d78-8502-45557526ea49", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "8c16dc28-690d-4d7e-aeb8-ee937ab86528"}], "security_groups": [{"name": "tempest-securitygroup--1752087237"}], "key_name": "tempest-keypair-1741077166", "user_data": "}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:34:00.939461 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.network.neutron [None req-54383ccd-c832-4691-abf0-a1d56322e1fc tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] validate_networks() for [('8c16dc28-690d-4d7e-aeb8-ee937ab86528', None, None, None, None, None)] {{(pid=127602) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 14:34:01.314107 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-54383ccd-c832-4691-abf0-a1d56322e1fc tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Flavor limits 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 14:34:01.314576 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-54383ccd-c832-4691-abf0-a1d56322e1fc tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Image limits 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 14:34:01.315075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-54383ccd-c832-4691-abf0-a1d56322e1fc tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Flavor pref 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 14:34:01.315483 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-54383ccd-c832-4691-abf0-a1d56322e1fc tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Image pref 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 14:34:01.316020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-54383ccd-c832-4691-abf0-a1d56322e1fc tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 14:34:01.324931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-54383ccd-c832-4691-abf0-a1d56322e1fc tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:01.325415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-54383ccd-c832-4691-abf0-a1d56322e1fc tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:01.325845 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-54383ccd-c832-4691-abf0-a1d56322e1fc tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:01.326604 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-54383ccd-c832-4691-abf0-a1d56322e1fc tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:01.327054 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-54383ccd-c832-4691-abf0-a1d56322e1fc tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:01.327475 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-54383ccd-c832-4691-abf0-a1d56322e1fc tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:01.344942 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-54383ccd-c832-4691-abf0-a1d56322e1fc tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Getting quotas for project b21f666643b74b5cba0b7597ae1aa0cd. Resources: {'cores', 'instances', 'ram'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 14:34:01.349582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-54383ccd-c832-4691-abf0-a1d56322e1fc tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Getting quotas for user 357ebb86fd664e08b15bf440e2a61560 and project b21f666643b74b5cba0b7597ae1aa0cd. Resources: {'cores', 'instances', 'ram'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 14:34:01.361019 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-54383ccd-c832-4691-abf0-a1d56322e1fc tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Going to run 1 instances... {{(pid=127602) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 14:34:01.363369 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-54383ccd-c832-4691-abf0-a1d56322e1fc tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='20b30425-92e5-4d78-8502-45557526ea49',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127602) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 14:34:01.517290 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-54383ccd-c832-4691-abf0-a1d56322e1fc tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.689229 Sep 08 14:34:01.517870 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1264/2810] 10.222.0.25 () {60 vars in 1147 bytes} [Thu Sep 8 14:34:00 2022] POST /compute/v2.1/servers => generated 402 bytes in 691 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 08 14:34:01.529404 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-eb3e7c3c-010d-49b9-a2e3-0481bf9f7dc4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:34:01.530030 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-eb3e7c3c-010d-49b9-a2e3-0481bf9f7dc4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:34:01.602272 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-eb3e7c3c-010d-49b9-a2e3-0481bf9f7dc4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:34:01.613823 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-eb3e7c3c-010d-49b9-a2e3-0481bf9f7dc4 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 200 len: 1236 microversion: 2.1 time: 0.088316 Sep 08 14:34:01.615043 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1547/2811] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:34:01 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 1236 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:34:02.637336 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-94c8fb64-2b73-4607-b771-c28c68f17718 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:34:02.637942 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-94c8fb64-2b73-4607-b771-c28c68f17718 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:34:02.645300 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94c8fb64-2b73-4607-b771-c28c68f17718 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:02.646304 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94c8fb64-2b73-4607-b771-c28c68f17718 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:02.647069 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94c8fb64-2b73-4607-b771-c28c68f17718 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:02.647428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94c8fb64-2b73-4607-b771-c28c68f17718 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:02.647872 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94c8fb64-2b73-4607-b771-c28c68f17718 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:02.648288 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-94c8fb64-2b73-4607-b771-c28c68f17718 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:02.713696 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-94c8fb64-2b73-4607-b771-c28c68f17718 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:34:02.721009 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-94c8fb64-2b73-4607-b771-c28c68f17718 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 200 len: 1240 microversion: 2.1 time: 0.088018 Sep 08 14:34:02.721575 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1265/2812] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:34:02 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 1240 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:34:03.744631 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-84946b2f-5904-4945-8809-2654dc29d110 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:34:03.745425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-84946b2f-5904-4945-8809-2654dc29d110 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:34:03.752033 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84946b2f-5904-4945-8809-2654dc29d110 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:03.752546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84946b2f-5904-4945-8809-2654dc29d110 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:03.752983 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84946b2f-5904-4945-8809-2654dc29d110 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:03.753684 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84946b2f-5904-4945-8809-2654dc29d110 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:03.754152 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84946b2f-5904-4945-8809-2654dc29d110 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:03.754587 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-84946b2f-5904-4945-8809-2654dc29d110 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:03.828853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-84946b2f-5904-4945-8809-2654dc29d110 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:34:03.841580 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-84946b2f-5904-4945-8809-2654dc29d110 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 200 len: 1296 microversion: 2.1 time: 0.100864 Sep 08 14:34:03.842385 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1548/2813] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:34:03 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 1296 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:34:04.865836 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bd4ed1d6-4238-42b1-85d9-7f0d8a7d053f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:34:04.866557 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bd4ed1d6-4238-42b1-85d9-7f0d8a7d053f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:34:04.872876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd4ed1d6-4238-42b1-85d9-7f0d8a7d053f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:04.873371 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd4ed1d6-4238-42b1-85d9-7f0d8a7d053f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:04.873812 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd4ed1d6-4238-42b1-85d9-7f0d8a7d053f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:04.874571 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd4ed1d6-4238-42b1-85d9-7f0d8a7d053f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:04.875032 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd4ed1d6-4238-42b1-85d9-7f0d8a7d053f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:04.875457 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bd4ed1d6-4238-42b1-85d9-7f0d8a7d053f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:05.033624 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-bd4ed1d6-4238-42b1-85d9-7f0d8a7d053f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:34:05.044450 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bd4ed1d6-4238-42b1-85d9-7f0d8a7d053f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 200 len: 1362 microversion: 2.1 time: 0.183223 Sep 08 14:34:05.046243 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1266/2814] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:34:04 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 1362 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:34:05.627182 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-c6049891-391d-403c-adaf-d28ecf2a19d0 req-e24d681b-3e2d-4931-a1b1-f8164c081139 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f7ffe8c2-a406-42d2-b8ba-89bf43352f83", "tag": "6ec666d3-e9f2-4ccd-9446-aef8eb2114b1"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:34:05.638840 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-c6049891-391d-403c-adaf-d28ecf2a19d0 req-e24d681b-3e2d-4931-a1b1-f8164c081139 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:05.639998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-c6049891-391d-403c-adaf-d28ecf2a19d0 req-e24d681b-3e2d-4931-a1b1-f8164c081139 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:05.639998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-c6049891-391d-403c-adaf-d28ecf2a19d0 req-e24d681b-3e2d-4931-a1b1-f8164c081139 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:05.654582 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-c6049891-391d-403c-adaf-d28ecf2a19d0 req-e24d681b-3e2d-4931-a1b1-f8164c081139 service nova] Creating event network-changed:6ec666d3-e9f2-4ccd-9446-aef8eb2114b1 for instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 on hv-cinder-85646 Sep 08 14:34:05.659636 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-c6049891-391d-403c-adaf-d28ecf2a19d0 req-e24d681b-3e2d-4931-a1b1-f8164c081139 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036259 Sep 08 14:34:05.660283 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1549/2815] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:34:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:34:06.069172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-09c52a9d-d655-4166-a30d-3d0fab54dd51 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:34:06.069780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-09c52a9d-d655-4166-a30d-3d0fab54dd51 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:34:06.076577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-09c52a9d-d655-4166-a30d-3d0fab54dd51 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:06.077091 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-09c52a9d-d655-4166-a30d-3d0fab54dd51 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:06.077519 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-09c52a9d-d655-4166-a30d-3d0fab54dd51 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:06.078225 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-09c52a9d-d655-4166-a30d-3d0fab54dd51 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:06.078672 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-09c52a9d-d655-4166-a30d-3d0fab54dd51 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:06.079094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-09c52a9d-d655-4166-a30d-3d0fab54dd51 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:06.214999 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-09c52a9d-d655-4166-a30d-3d0fab54dd51 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:34:06.222932 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-09c52a9d-d655-4166-a30d-3d0fab54dd51 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 200 len: 1362 microversion: 2.1 time: 0.157698 Sep 08 14:34:06.224496 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1267/2816] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:34:06 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 1362 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:34:07.250507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bcb0c200-dc66-4448-93ec-d417e6cc098f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:34:07.250507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bcb0c200-dc66-4448-93ec-d417e6cc098f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:34:07.254469 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcb0c200-dc66-4448-93ec-d417e6cc098f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:07.255629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcb0c200-dc66-4448-93ec-d417e6cc098f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:07.256081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcb0c200-dc66-4448-93ec-d417e6cc098f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:07.256788 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcb0c200-dc66-4448-93ec-d417e6cc098f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:07.257240 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcb0c200-dc66-4448-93ec-d417e6cc098f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:07.257671 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bcb0c200-dc66-4448-93ec-d417e6cc098f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:07.389229 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-bcb0c200-dc66-4448-93ec-d417e6cc098f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:34:07.397149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bcb0c200-dc66-4448-93ec-d417e6cc098f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 200 len: 1362 microversion: 2.1 time: 0.154706 Sep 08 14:34:07.397705 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1550/2817] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:34:07 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 1362 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:34:08.421817 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c60247ad-7166-48ae-82a8-c30dbb2aa530 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:34:08.422457 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c60247ad-7166-48ae-82a8-c30dbb2aa530 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:34:08.429678 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c60247ad-7166-48ae-82a8-c30dbb2aa530 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:08.430211 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c60247ad-7166-48ae-82a8-c30dbb2aa530 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:08.430613 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c60247ad-7166-48ae-82a8-c30dbb2aa530 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:08.431303 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c60247ad-7166-48ae-82a8-c30dbb2aa530 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:08.431742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c60247ad-7166-48ae-82a8-c30dbb2aa530 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:08.432165 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c60247ad-7166-48ae-82a8-c30dbb2aa530 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:08.561816 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c60247ad-7166-48ae-82a8-c30dbb2aa530 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:34:08.569434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c60247ad-7166-48ae-82a8-c30dbb2aa530 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 200 len: 1362 microversion: 2.1 time: 0.152719 Sep 08 14:34:08.569989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1268/2818] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:34:08 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 1362 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:34:09.593029 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-665b3a87-bf1e-4349-8b71-b95c2a07ac69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:34:09.593629 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-665b3a87-bf1e-4349-8b71-b95c2a07ac69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:34:09.600267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-665b3a87-bf1e-4349-8b71-b95c2a07ac69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:09.600752 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-665b3a87-bf1e-4349-8b71-b95c2a07ac69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:09.601233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-665b3a87-bf1e-4349-8b71-b95c2a07ac69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:09.601976 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-665b3a87-bf1e-4349-8b71-b95c2a07ac69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:09.602467 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-665b3a87-bf1e-4349-8b71-b95c2a07ac69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:09.602894 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-665b3a87-bf1e-4349-8b71-b95c2a07ac69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:09.738045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-665b3a87-bf1e-4349-8b71-b95c2a07ac69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:34:09.746945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-665b3a87-bf1e-4349-8b71-b95c2a07ac69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 200 len: 1362 microversion: 2.1 time: 0.157892 Sep 08 14:34:09.747581 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1551/2819] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:34:09 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 1362 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:34:10.769840 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-bebddd56-36b0-4be2-bf2a-ef8a510c8f24 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:34:10.770439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-bebddd56-36b0-4be2-bf2a-ef8a510c8f24 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:34:10.777518 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bebddd56-36b0-4be2-bf2a-ef8a510c8f24 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:10.778003 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bebddd56-36b0-4be2-bf2a-ef8a510c8f24 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:10.778470 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bebddd56-36b0-4be2-bf2a-ef8a510c8f24 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:10.779162 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bebddd56-36b0-4be2-bf2a-ef8a510c8f24 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:10.779652 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bebddd56-36b0-4be2-bf2a-ef8a510c8f24 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:10.780139 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-bebddd56-36b0-4be2-bf2a-ef8a510c8f24 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:11.159534 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-bebddd56-36b0-4be2-bf2a-ef8a510c8f24 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:34:11.167947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-bebddd56-36b0-4be2-bf2a-ef8a510c8f24 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 200 len: 1362 microversion: 2.1 time: 0.402313 Sep 08 14:34:11.168522 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1269/2820] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:34:10 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 1362 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:34:11.477602 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-d07460f9-6d2c-4297-82f8-4377ec415ce6 req-b58ead25-40a8-4730-b35b-0711ed40bd93 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f7ffe8c2-a406-42d2-b8ba-89bf43352f83", "name": "network-vif-plugged", "status": "completed", "tag": "6ec666d3-e9f2-4ccd-9446-aef8eb2114b1"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:34:11.487553 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-d07460f9-6d2c-4297-82f8-4377ec415ce6 req-b58ead25-40a8-4730-b35b-0711ed40bd93 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:11.488031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-d07460f9-6d2c-4297-82f8-4377ec415ce6 req-b58ead25-40a8-4730-b35b-0711ed40bd93 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:11.488462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-d07460f9-6d2c-4297-82f8-4377ec415ce6 req-b58ead25-40a8-4730-b35b-0711ed40bd93 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:11.500163 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-d07460f9-6d2c-4297-82f8-4377ec415ce6 req-b58ead25-40a8-4730-b35b-0711ed40bd93 service nova] Creating event network-vif-plugged:6ec666d3-e9f2-4ccd-9446-aef8eb2114b1 for instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 on hv-cinder-85646 Sep 08 14:34:11.505067 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-d07460f9-6d2c-4297-82f8-4377ec415ce6 req-b58ead25-40a8-4730-b35b-0711ed40bd93 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031846 Sep 08 14:34:11.505660 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1552/2821] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:34:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:34:12.190795 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d00063e1-95c8-4810-a51c-ed658071fad1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:34:12.191399 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d00063e1-95c8-4810-a51c-ed658071fad1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:34:12.193995 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=2 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:34:12.199219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=6 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:34:12.199876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d00063e1-95c8-4810-a51c-ed658071fad1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:12.200348 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d00063e1-95c8-4810-a51c-ed658071fad1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:12.200772 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d00063e1-95c8-4810-a51c-ed658071fad1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:12.201476 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d00063e1-95c8-4810-a51c-ed658071fad1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:12.201919 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d00063e1-95c8-4810-a51c-ed658071fad1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:12.202379 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d00063e1-95c8-4810-a51c-ed658071fad1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:12.331637 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-d00063e1-95c8-4810-a51c-ed658071fad1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:34:12.339313 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d00063e1-95c8-4810-a51c-ed658071fad1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 200 len: 1362 microversion: 2.1 time: 0.152520 Sep 08 14:34:12.339866 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1270/2822] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:34:12 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 1362 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:34:13.359695 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=6 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:34:13.364471 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-81c8c696-8026-46f3-881c-e03f072874b5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:34:13.365178 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-81c8c696-8026-46f3-881c-e03f072874b5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:34:13.371419 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81c8c696-8026-46f3-881c-e03f072874b5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:13.371911 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81c8c696-8026-46f3-881c-e03f072874b5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:13.372355 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81c8c696-8026-46f3-881c-e03f072874b5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:13.373049 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81c8c696-8026-46f3-881c-e03f072874b5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:13.373529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81c8c696-8026-46f3-881c-e03f072874b5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:13.373938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81c8c696-8026-46f3-881c-e03f072874b5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:13.515904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-81c8c696-8026-46f3-881c-e03f072874b5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:34:13.524399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-81c8c696-8026-46f3-881c-e03f072874b5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 200 len: 1362 microversion: 2.1 time: 0.165684 Sep 08 14:34:13.524964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1553/2823] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:34:13 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 1362 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:34:14.547100 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c92a51bc-c0a7-4c1e-ab9b-839b20751ae1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:34:14.547709 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c92a51bc-c0a7-4c1e-ab9b-839b20751ae1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:34:14.554961 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c92a51bc-c0a7-4c1e-ab9b-839b20751ae1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:14.555440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c92a51bc-c0a7-4c1e-ab9b-839b20751ae1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:14.555897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c92a51bc-c0a7-4c1e-ab9b-839b20751ae1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:14.556593 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c92a51bc-c0a7-4c1e-ab9b-839b20751ae1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:14.557041 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c92a51bc-c0a7-4c1e-ab9b-839b20751ae1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:14.557459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c92a51bc-c0a7-4c1e-ab9b-839b20751ae1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:14.689479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c92a51bc-c0a7-4c1e-ab9b-839b20751ae1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:34:14.697986 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c92a51bc-c0a7-4c1e-ab9b-839b20751ae1 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 200 len: 1548 microversion: 2.1 time: 0.154897 Sep 08 14:34:14.698592 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1271/2824] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:34:14 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 1548 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:34:14.720796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-031024e5-d54f-440b-bb39-805b8e45a540 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:34:14.721281 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-031024e5-d54f-440b-bb39-805b8e45a540 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:34:14.727899 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-031024e5-d54f-440b-bb39-805b8e45a540 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:14.728394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-031024e5-d54f-440b-bb39-805b8e45a540 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:14.728834 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-031024e5-d54f-440b-bb39-805b8e45a540 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:14.729562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-031024e5-d54f-440b-bb39-805b8e45a540 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:14.730020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-031024e5-d54f-440b-bb39-805b8e45a540 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:14.730489 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-031024e5-d54f-440b-bb39-805b8e45a540 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:14.812523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-031024e5-d54f-440b-bb39-805b8e45a540 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.095561 Sep 08 14:34:14.813142 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1554/2825] 10.222.0.25 () {58 vars in 1275 bytes} [Thu Sep 8 14:34:14 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-interface => generated 285 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:34:15.853773 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-f6c122e7-4290-42ec-ac9b-f56adcb59f73 req-c5106092-9ede-40ab-adb3-e3603343ed2e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f7ffe8c2-a406-42d2-b8ba-89bf43352f83", "tag": "6ec666d3-e9f2-4ccd-9446-aef8eb2114b1"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:34:15.865333 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-f6c122e7-4290-42ec-ac9b-f56adcb59f73 req-c5106092-9ede-40ab-adb3-e3603343ed2e service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:15.865802 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-f6c122e7-4290-42ec-ac9b-f56adcb59f73 req-c5106092-9ede-40ab-adb3-e3603343ed2e service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:15.866254 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-f6c122e7-4290-42ec-ac9b-f56adcb59f73 req-c5106092-9ede-40ab-adb3-e3603343ed2e service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:15.879921 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-f6c122e7-4290-42ec-ac9b-f56adcb59f73 req-c5106092-9ede-40ab-adb3-e3603343ed2e service nova] Creating event network-changed:6ec666d3-e9f2-4ccd-9446-aef8eb2114b1 for instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 on hv-cinder-85646 Sep 08 14:34:15.884962 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-f6c122e7-4290-42ec-ac9b-f56adcb59f73 req-c5106092-9ede-40ab-adb3-e3603343ed2e service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036821 Sep 08 14:34:15.885551 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1272/2826] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:34:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:34:52.064814 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=39,nova_api:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:34:52.065590 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=54 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:34:52.069766 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-432c6b60-5194-4343-853e-3bc8750d97f5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:34:52.070296 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-432c6b60-5194-4343-853e-3bc8750d97f5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:34:52.076476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-432c6b60-5194-4343-853e-3bc8750d97f5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:52.078095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-432c6b60-5194-4343-853e-3bc8750d97f5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:52.078095 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-432c6b60-5194-4343-853e-3bc8750d97f5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:52.078410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-432c6b60-5194-4343-853e-3bc8750d97f5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:52.078767 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-432c6b60-5194-4343-853e-3bc8750d97f5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:52.079206 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-432c6b60-5194-4343-853e-3bc8750d97f5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:52.103225 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-432c6b60-5194-4343-853e-3bc8750d97f5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/ips" status: 200 len: 146 microversion: 2.1 time: 0.039721 Sep 08 14:34:52.103798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1555/2827] 10.222.0.25 () {58 vars in 1248 bytes} [Thu Sep 8 14:34:52 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/ips => generated 146 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:34:53.733510 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=49,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:34:53.734292 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=80,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:34:53.738732 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a712ef56-d596-4d02-9e15-50546258a40e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "ce06d898-6451-414a-8f2a-928d4e1f62b2"}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:34:53.741157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a712ef56-d596-4d02-9e15-50546258a40e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:34:53.747493 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a712ef56-d596-4d02-9e15-50546258a40e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:53.748007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a712ef56-d596-4d02-9e15-50546258a40e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:53.748428 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a712ef56-d596-4d02-9e15-50546258a40e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:53.749625 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a712ef56-d596-4d02-9e15-50546258a40e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:34:53.750077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a712ef56-d596-4d02-9e15-50546258a40e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:34:53.750525 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a712ef56-d596-4d02-9e15-50546258a40e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:34:54.416409 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a712ef56-d596-4d02-9e15-50546258a40e tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "POST /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.684229 Sep 08 14:34:54.417026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1273/2828] 10.222.0.25 () {60 vars in 1323 bytes} [Thu Sep 8 14:34:53 2022] POST /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments => generated 194 bytes in 686 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:35:01.974670 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3602f05d-9841-429c-bff2-c5f632a450fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:01.976810 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3602f05d-9841-429c-bff2-c5f632a450fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:01.983225 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3602f05d-9841-429c-bff2-c5f632a450fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:01.983712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3602f05d-9841-429c-bff2-c5f632a450fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:01.984153 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3602f05d-9841-429c-bff2-c5f632a450fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:01.984837 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3602f05d-9841-429c-bff2-c5f632a450fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:01.985294 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3602f05d-9841-429c-bff2-c5f632a450fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:01.985721 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3602f05d-9841-429c-bff2-c5f632a450fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:02.016786 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3602f05d-9841-429c-bff2-c5f632a450fa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046886 Sep 08 14:35:02.017934 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1556/2829] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:35:01 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:35:02.029466 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1a08b263-3aef-4d25-8d29-82e22b69ee60 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:02.030021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1a08b263-3aef-4d25-8d29-82e22b69ee60 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:02.036567 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1a08b263-3aef-4d25-8d29-82e22b69ee60 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:02.037042 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1a08b263-3aef-4d25-8d29-82e22b69ee60 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:02.037471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1a08b263-3aef-4d25-8d29-82e22b69ee60 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:02.038164 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1a08b263-3aef-4d25-8d29-82e22b69ee60 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:02.038611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1a08b263-3aef-4d25-8d29-82e22b69ee60 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:02.039024 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1a08b263-3aef-4d25-8d29-82e22b69ee60 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:02.067144 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1a08b263-3aef-4d25-8d29-82e22b69ee60 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments/ce06d898-6451-414a-8f2a-928d4e1f62b2" status: 200 len: 194 microversion: 2.1 time: 0.042056 Sep 08 14:35:02.067707 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1274/2830] 10.222.0.25 () {58 vars in 1413 bytes} [Thu Sep 8 14:35:02 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments/ce06d898-6451-414a-8f2a-928d4e1f62b2 => generated 194 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:35:03.627136 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-db30e06b-761e-4be2-a5f1-412b45a50906 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "20a466b2-a0f1-42bc-ad2b-1937d543a013"}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:35:03.629444 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-db30e06b-761e-4be2-a5f1-412b45a50906 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:03.637498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db30e06b-761e-4be2-a5f1-412b45a50906 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:03.637498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db30e06b-761e-4be2-a5f1-412b45a50906 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:03.637498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db30e06b-761e-4be2-a5f1-412b45a50906 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:03.637844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db30e06b-761e-4be2-a5f1-412b45a50906 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:03.638276 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db30e06b-761e-4be2-a5f1-412b45a50906 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:03.638709 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-db30e06b-761e-4be2-a5f1-412b45a50906 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:04.375897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-db30e06b-761e-4be2-a5f1-412b45a50906 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "POST /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.752577 Sep 08 14:35:04.377130 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1557/2831] 10.222.0.25 () {60 vars in 1323 bytes} [Thu Sep 8 14:35:03 2022] POST /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments => generated 194 bytes in 755 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:35:12.988858 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cf412328-608b-4b67-8ca7-0e5abc0e4f69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:12.991079 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cf412328-608b-4b67-8ca7-0e5abc0e4f69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:12.993607 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=4 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:12.994300 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=17,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:35:12.999136 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cf412328-608b-4b67-8ca7-0e5abc0e4f69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:12.999618 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cf412328-608b-4b67-8ca7-0e5abc0e4f69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:13.000052 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cf412328-608b-4b67-8ca7-0e5abc0e4f69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:13.000742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cf412328-608b-4b67-8ca7-0e5abc0e4f69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:13.001191 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cf412328-608b-4b67-8ca7-0e5abc0e4f69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:13.001617 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cf412328-608b-4b67-8ca7-0e5abc0e4f69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:13.031628 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cf412328-608b-4b67-8ca7-0e5abc0e4f69 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.047381 Sep 08 14:35:13.032224 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1275/2832] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:35:12 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments => generated 371 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:35:13.179031 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3808a445-9af7-4192-9a76-1bf8a311db3f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:13.179607 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3808a445-9af7-4192-9a76-1bf8a311db3f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:13.186068 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3808a445-9af7-4192-9a76-1bf8a311db3f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:13.186606 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3808a445-9af7-4192-9a76-1bf8a311db3f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:13.187052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3808a445-9af7-4192-9a76-1bf8a311db3f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:13.187751 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3808a445-9af7-4192-9a76-1bf8a311db3f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:13.188245 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3808a445-9af7-4192-9a76-1bf8a311db3f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:13.188677 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3808a445-9af7-4192-9a76-1bf8a311db3f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:13.217534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3808a445-9af7-4192-9a76-1bf8a311db3f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments/ce06d898-6451-414a-8f2a-928d4e1f62b2" status: 200 len: 194 microversion: 2.1 time: 0.043899 Sep 08 14:35:13.218122 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1558/2833] 10.222.0.25 () {58 vars in 1413 bytes} [Thu Sep 8 14:35:13 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments/ce06d898-6451-414a-8f2a-928d4e1f62b2 => generated 194 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:35:13.229555 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2b2c4ac7-3292-4e82-8a4c-68db6de53fbf tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:13.230046 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2b2c4ac7-3292-4e82-8a4c-68db6de53fbf tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:13.236606 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b2c4ac7-3292-4e82-8a4c-68db6de53fbf tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:13.237089 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b2c4ac7-3292-4e82-8a4c-68db6de53fbf tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:13.238209 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b2c4ac7-3292-4e82-8a4c-68db6de53fbf tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:13.238963 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b2c4ac7-3292-4e82-8a4c-68db6de53fbf tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:13.239427 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b2c4ac7-3292-4e82-8a4c-68db6de53fbf tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:13.239852 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2b2c4ac7-3292-4e82-8a4c-68db6de53fbf tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:13.321120 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-2b2c4ac7-3292-4e82-8a4c-68db6de53fbf tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lazy-loading 'services' on Instance uuid f7ffe8c2-a406-42d2-b8ba-89bf43352f83 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:35:13.560691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2b2c4ac7-3292-4e82-8a4c-68db6de53fbf tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "DELETE /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments/ce06d898-6451-414a-8f2a-928d4e1f62b2" status: 202 len: 0 microversion: 2.1 time: 0.335377 Sep 08 14:35:13.561362 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1276/2834] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:35:13 2022] DELETE /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments/ce06d898-6451-414a-8f2a-928d4e1f62b2 => generated 0 bytes in 337 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:35:15.741298 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-bace4313-2e8b-4a09-8cc6-00279fa96873 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:15.741810 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-bace4313-2e8b-4a09-8cc6-00279fa96873 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:15.748346 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bace4313-2e8b-4a09-8cc6-00279fa96873 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:15.748815 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bace4313-2e8b-4a09-8cc6-00279fa96873 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:15.749247 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bace4313-2e8b-4a09-8cc6-00279fa96873 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:15.749927 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bace4313-2e8b-4a09-8cc6-00279fa96873 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:15.750407 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bace4313-2e8b-4a09-8cc6-00279fa96873 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:15.750831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-bace4313-2e8b-4a09-8cc6-00279fa96873 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:15.780560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-bace4313-2e8b-4a09-8cc6-00279fa96873 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments/20a466b2-a0f1-42bc-ad2b-1937d543a013" status: 200 len: 194 microversion: 2.1 time: 0.043635 Sep 08 14:35:15.781181 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1559/2835] 10.222.0.25 () {58 vars in 1413 bytes} [Thu Sep 8 14:35:15 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments/20a466b2-a0f1-42bc-ad2b-1937d543a013 => generated 194 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:35:15.793464 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f0d63257-3164-4e1f-b764-97125667c71d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:15.794078 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f0d63257-3164-4e1f-b764-97125667c71d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:15.801437 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f0d63257-3164-4e1f-b764-97125667c71d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:15.801973 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f0d63257-3164-4e1f-b764-97125667c71d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:15.802514 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f0d63257-3164-4e1f-b764-97125667c71d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:15.803306 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f0d63257-3164-4e1f-b764-97125667c71d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:15.803798 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f0d63257-3164-4e1f-b764-97125667c71d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:15.804216 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f0d63257-3164-4e1f-b764-97125667c71d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:15.888760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-f0d63257-3164-4e1f-b764-97125667c71d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lazy-loading 'services' on Instance uuid f7ffe8c2-a406-42d2-b8ba-89bf43352f83 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:35:16.080313 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f0d63257-3164-4e1f-b764-97125667c71d tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "DELETE /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments/20a466b2-a0f1-42bc-ad2b-1937d543a013" status: 202 len: 0 microversion: 2.1 time: 0.291862 Sep 08 14:35:16.080908 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1277/2836] 10.222.0.25 () {58 vars in 1416 bytes} [Thu Sep 8 14:35:15 2022] DELETE /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments/20a466b2-a0f1-42bc-ad2b-1937d543a013 => generated 0 bytes in 294 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:35:17.257715 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f86ae135-1e38-4da8-a197-de3e05fbe967 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:17.259091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f86ae135-1e38-4da8-a197-de3e05fbe967 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:17.265867 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f86ae135-1e38-4da8-a197-de3e05fbe967 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:17.266393 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f86ae135-1e38-4da8-a197-de3e05fbe967 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:17.266835 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f86ae135-1e38-4da8-a197-de3e05fbe967 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:17.267534 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f86ae135-1e38-4da8-a197-de3e05fbe967 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:17.267984 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f86ae135-1e38-4da8-a197-de3e05fbe967 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:17.268406 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f86ae135-1e38-4da8-a197-de3e05fbe967 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:17.298342 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f86ae135-1e38-4da8-a197-de3e05fbe967 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.046186 Sep 08 14:35:17.298901 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1560/2837] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:35:17 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments => generated 25 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 08 14:35:17.424115 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-aa7241f7-afbb-4b40-8062-2f5020e86db6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:17.427200 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-aa7241f7-afbb-4b40-8062-2f5020e86db6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:17.433471 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa7241f7-afbb-4b40-8062-2f5020e86db6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:17.433946 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa7241f7-afbb-4b40-8062-2f5020e86db6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:17.434397 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa7241f7-afbb-4b40-8062-2f5020e86db6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:17.435077 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa7241f7-afbb-4b40-8062-2f5020e86db6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:17.435520 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa7241f7-afbb-4b40-8062-2f5020e86db6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:17.435941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-aa7241f7-afbb-4b40-8062-2f5020e86db6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:17.469702 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-aa7241f7-afbb-4b40-8062-2f5020e86db6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.049988 Sep 08 14:35:17.469982 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1278/2838] 10.222.0.25 () {58 vars in 1302 bytes} [Thu Sep 8 14:35:17 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83/os-volume_attachments => generated 25 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 08 14:35:17.537211 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-68c2f990-33b6-4bdb-ac52-e432362f70bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:17.537710 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-68c2f990-33b6-4bdb-ac52-e432362f70bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:17.543738 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68c2f990-33b6-4bdb-ac52-e432362f70bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:17.544236 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68c2f990-33b6-4bdb-ac52-e432362f70bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:17.544678 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68c2f990-33b6-4bdb-ac52-e432362f70bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:17.545368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68c2f990-33b6-4bdb-ac52-e432362f70bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:17.545830 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68c2f990-33b6-4bdb-ac52-e432362f70bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:17.546300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-68c2f990-33b6-4bdb-ac52-e432362f70bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:17.571476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-68c2f990-33b6-4bdb-ac52-e432362f70bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:35:17.817300 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-68c2f990-33b6-4bdb-ac52-e432362f70bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "DELETE /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 204 len: 0 microversion: 2.1 time: 0.284119 Sep 08 14:35:17.817967 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1561/2839] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:35:17 2022] DELETE /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:35:17.827120 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-64cdb688-e354-4799-8827-8c8f5f407553 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:17.827881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-64cdb688-e354-4799-8827-8c8f5f407553 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:17.835797 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64cdb688-e354-4799-8827-8c8f5f407553 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:17.836543 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64cdb688-e354-4799-8827-8c8f5f407553 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:17.836973 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64cdb688-e354-4799-8827-8c8f5f407553 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:17.837719 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64cdb688-e354-4799-8827-8c8f5f407553 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:17.838351 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64cdb688-e354-4799-8827-8c8f5f407553 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:17.838769 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-64cdb688-e354-4799-8827-8c8f5f407553 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:17.977029 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-64cdb688-e354-4799-8827-8c8f5f407553 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:35:17.984787 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-64cdb688-e354-4799-8827-8c8f5f407553 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 200 len: 1673 microversion: 2.1 time: 0.162499 Sep 08 14:35:17.985356 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1279/2840] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:35:17 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 1673 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:35:19.009183 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9d96b1c0-e7ce-421b-9cab-5ee0538deffa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:19.009895 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9d96b1c0-e7ce-421b-9cab-5ee0538deffa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:19.016264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9d96b1c0-e7ce-421b-9cab-5ee0538deffa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:19.016757 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9d96b1c0-e7ce-421b-9cab-5ee0538deffa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:19.017202 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9d96b1c0-e7ce-421b-9cab-5ee0538deffa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:19.017918 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9d96b1c0-e7ce-421b-9cab-5ee0538deffa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:19.018405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9d96b1c0-e7ce-421b-9cab-5ee0538deffa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:19.018844 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9d96b1c0-e7ce-421b-9cab-5ee0538deffa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:19.162641 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9d96b1c0-e7ce-421b-9cab-5ee0538deffa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:35:19.170546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9d96b1c0-e7ce-421b-9cab-5ee0538deffa tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 200 len: 1673 microversion: 2.1 time: 0.165584 Sep 08 14:35:19.171126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1562/2841] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:35:19 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 1673 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:35:20.196574 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7650b278-2ec3-4ad2-bda5-b0e5a515cf91 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:20.197230 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7650b278-2ec3-4ad2-bda5-b0e5a515cf91 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:20.203589 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7650b278-2ec3-4ad2-bda5-b0e5a515cf91 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:20.204065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7650b278-2ec3-4ad2-bda5-b0e5a515cf91 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:20.204492 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7650b278-2ec3-4ad2-bda5-b0e5a515cf91 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:20.205168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7650b278-2ec3-4ad2-bda5-b0e5a515cf91 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:20.205606 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7650b278-2ec3-4ad2-bda5-b0e5a515cf91 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:20.206040 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7650b278-2ec3-4ad2-bda5-b0e5a515cf91 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:20.336474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7650b278-2ec3-4ad2-bda5-b0e5a515cf91 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:35:20.344273 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7650b278-2ec3-4ad2-bda5-b0e5a515cf91 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 200 len: 1673 microversion: 2.1 time: 0.151531 Sep 08 14:35:20.344848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1280/2842] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:35:20 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 1673 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:35:21.367935 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4f4bca99-9174-479c-8116-8284df4c835c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:21.368543 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4f4bca99-9174-479c-8116-8284df4c835c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:21.375005 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f4bca99-9174-479c-8116-8284df4c835c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:21.375491 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f4bca99-9174-479c-8116-8284df4c835c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:21.381700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f4bca99-9174-479c-8116-8284df4c835c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:21.381700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f4bca99-9174-479c-8116-8284df4c835c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:21.381700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f4bca99-9174-479c-8116-8284df4c835c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:21.381700 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4f4bca99-9174-479c-8116-8284df4c835c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:21.454019 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4f4bca99-9174-479c-8116-8284df4c835c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:35:21.462723 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4f4bca99-9174-479c-8116-8284df4c835c tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 200 len: 1605 microversion: 2.1 time: 0.098535 Sep 08 14:35:21.474247 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1563/2843] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:35:21 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 1605 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:35:21.698222 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-514c99ab-3d03-4d82-a606-33004048817c req-93be09d9-119c-427e-8720-692c3df9efe6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f7ffe8c2-a406-42d2-b8ba-89bf43352f83", "name": "network-vif-deleted", "tag": "6ec666d3-e9f2-4ccd-9446-aef8eb2114b1"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:35:21.712097 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-514c99ab-3d03-4d82-a606-33004048817c req-93be09d9-119c-427e-8720-692c3df9efe6 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:21.712785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-514c99ab-3d03-4d82-a606-33004048817c req-93be09d9-119c-427e-8720-692c3df9efe6 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:21.713386 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-514c99ab-3d03-4d82-a606-33004048817c req-93be09d9-119c-427e-8720-692c3df9efe6 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:21.726538 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-514c99ab-3d03-4d82-a606-33004048817c req-93be09d9-119c-427e-8720-692c3df9efe6 service nova] Creating event network-vif-deleted:6ec666d3-e9f2-4ccd-9446-aef8eb2114b1 for instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 on hv-cinder-85646 Sep 08 14:35:21.732416 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-514c99ab-3d03-4d82-a606-33004048817c req-93be09d9-119c-427e-8720-692c3df9efe6 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.147928 Sep 08 14:35:21.733406 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1281/2844] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:35:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:35:22.489267 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-5b1daeae-6137-41f6-8947-c3b4631e7b0f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:22.489915 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-5b1daeae-6137-41f6-8947-c3b4631e7b0f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:22.496420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b1daeae-6137-41f6-8947-c3b4631e7b0f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:22.496957 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b1daeae-6137-41f6-8947-c3b4631e7b0f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:22.497349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b1daeae-6137-41f6-8947-c3b4631e7b0f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:22.498055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b1daeae-6137-41f6-8947-c3b4631e7b0f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:22.498531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b1daeae-6137-41f6-8947-c3b4631e7b0f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:22.498958 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b1daeae-6137-41f6-8947-c3b4631e7b0f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:22.531161 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b1daeae-6137-41f6-8947-c3b4631e7b0f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:22.531652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b1daeae-6137-41f6-8947-c3b4631e7b0f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:22.532093 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-5b1daeae-6137-41f6-8947-c3b4631e7b0f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:22.532543 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.objects.instance [None req-5b1daeae-6137-41f6-8947-c3b4631e7b0f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lazy-loading 'fault' on Instance uuid f7ffe8c2-a406-42d2-b8ba-89bf43352f83 {{(pid=127601) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:35:22.588179 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-5b1daeae-6137-41f6-8947-c3b4631e7b0f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '357ebb86fd664e08b15bf440e2a61560', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b21f666643b74b5cba0b7597ae1aa0cd', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:35:22.599020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-5b1daeae-6137-41f6-8947-c3b4631e7b0f tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 200 len: 1326 microversion: 2.1 time: 0.113106 Sep 08 14:35:22.599717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1564/2845] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:35:22 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 1326 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:35:23.675734 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d0b7ad3f-17a6-4f6b-a623-54e5ccb47832 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:23.676545 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-d0b7ad3f-17a6-4f6b-a623-54e5ccb47832 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:23.682926 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d0b7ad3f-17a6-4f6b-a623-54e5ccb47832 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:23.683408 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d0b7ad3f-17a6-4f6b-a623-54e5ccb47832 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:23.683840 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d0b7ad3f-17a6-4f6b-a623-54e5ccb47832 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:23.684517 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d0b7ad3f-17a6-4f6b-a623-54e5ccb47832 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:23.684968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d0b7ad3f-17a6-4f6b-a623-54e5ccb47832 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:23.685390 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-d0b7ad3f-17a6-4f6b-a623-54e5ccb47832 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:23.698557 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-d0b7ad3f-17a6-4f6b-a623-54e5ccb47832 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] HTTP exception thrown: Instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 could not be found. Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d0b7ad3f-17a6-4f6b-a623-54e5ccb47832 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Request method failure captured: Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 HTTP/1.1 Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfx-IBlTeU8lpWwGKB_djClew3IfsnnMsqAlTIyJwZatK6rO97X1sIwHrl836QwkkBwsDAti13QWgcktV2H3C3H2iyUMQVmkZ2rvFbwK_qsxWVKARvFf1vGfBVG_NZnxLXL77ZYDqLGjePARIcvrYdLR_yhLC_R9BR-JLVq-P_eoaHc Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: b21f666643b74b5cba0b7597ae1aa0cd Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeTestJSON-790394194 Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:35:23.700178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:35:23.701533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:35:23.701533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeTestJSON-790394194 Sep 08 14:35:23.701533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: b21f666643b74b5cba0b7597ae1aa0cd Sep 08 14:35:23.701533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeTestJSON-790394194 Sep 08 14:35:23.701533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeTestJSON-790394194-project Sep 08 14:35:23.701533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:35:23.701533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:35:23.701533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 357ebb86fd664e08b15bf440e2a61560 Sep 08 14:35:23.701533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeTestJSON-790394194-project Sep 08 14:35:23.701533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:35:23.701533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 could not be found. Sep 08 14:35:23.701533 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'f7ffe8c2-a406-42d2-b8ba-89bf43352f83'} Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 could not be found. Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:35:23.702302 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:35:23.703459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:35:23.703459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:35:23.703459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:35:23.703459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:35:23.703459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:35:23.703459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:35:23.703459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:35:23.703459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 could not be found. Sep 08 14:35:23.703459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:35:23.703459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:35:23.703459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:35:23.703459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:35:23.703459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:35:23.703459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:35:23.703459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:35:23.703459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:35:23.703459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 could not be found. Sep 08 14:35:23.703459 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:35:23.704500 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-d0b7ad3f-17a6-4f6b-a623-54e5ccb47832 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Returning 404 to user: Instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:35:23.704500 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-d0b7ad3f-17a6-4f6b-a623-54e5ccb47832 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 404 len: 111 microversion: 2.1 time: 0.082735 Sep 08 14:35:23.704500 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1282/2846] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:35:23 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 111 bytes in 84 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:35:23.710658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-488470d0-6dcc-4653-ad9f-0c8a594ea5e6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '.version_select at 0x7fce01bc83a0>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:23.802670 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-488470d0-6dcc-4653-ad9f-0c8a594ea5e6 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1741077166" status: 202 len: 0 microversion: 2.1 time: 0.095840 Sep 08 14:35:23.803293 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1565/2847] 10.222.0.25 () {58 vars in 1221 bytes} [Thu Sep 8 14:35:23 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1741077166 => generated 0 bytes in 97 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 08 14:35:27.192853 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6beb6fed-e38d-4062-be40-644ce5261ba5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:27.194000 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6beb6fed-e38d-4062-be40-644ce5261ba5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:27.200629 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6beb6fed-e38d-4062-be40-644ce5261ba5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:27.201103 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6beb6fed-e38d-4062-be40-644ce5261ba5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:27.201531 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6beb6fed-e38d-4062-be40-644ce5261ba5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:27.202315 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6beb6fed-e38d-4062-be40-644ce5261ba5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:27.202769 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6beb6fed-e38d-4062-be40-644ce5261ba5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:27.203190 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6beb6fed-e38d-4062-be40-644ce5261ba5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:27.217311 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-6beb6fed-e38d-4062-be40-644ce5261ba5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] HTTP exception thrown: Instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 could not be found. Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6beb6fed-e38d-4062-be40-644ce5261ba5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Request method failure captured: Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 HTTP/1.1 Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfx-IBlTeU8lpWwGKB_djClew3IfsnnMsqAlTIyJwZatK6rO97X1sIwHrl836QwkkBwsDAti13QWgcktV2H3C3H2iyUMQVmkZ2rvFbwK_qsxWVKARvFf1vGfBVG_NZnxLXL77ZYDqLGjePARIcvrYdLR_yhLC_R9BR-JLVq-P_eoaHc Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: b21f666643b74b5cba0b7597ae1aa0cd Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeTestJSON-790394194 Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:35:27.218888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:35:27.220039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:35:27.220039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeTestJSON-790394194 Sep 08 14:35:27.220039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: b21f666643b74b5cba0b7597ae1aa0cd Sep 08 14:35:27.220039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeTestJSON-790394194 Sep 08 14:35:27.220039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeTestJSON-790394194-project Sep 08 14:35:27.220039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:35:27.220039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:35:27.220039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 357ebb86fd664e08b15bf440e2a61560 Sep 08 14:35:27.220039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeTestJSON-790394194-project Sep 08 14:35:27.220039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:35:27.220039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 could not be found. Sep 08 14:35:27.220039 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'f7ffe8c2-a406-42d2-b8ba-89bf43352f83'} Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 could not be found. Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:35:27.220727 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 could not be found. Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 could not be found. Sep 08 14:35:27.222025 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:35:27.223265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6beb6fed-e38d-4062-be40-644ce5261ba5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Returning 404 to user: Instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:35:27.223265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6beb6fed-e38d-4062-be40-644ce5261ba5 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "DELETE /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 404 len: 111 microversion: 2.1 time: 0.031311 Sep 08 14:35:27.223265 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1283/2848] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:35:27 2022] DELETE /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:35:27.229394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a9aa1e8b-cc84-43a1-a07c-0fe32b55f9bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:27.230037 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-a9aa1e8b-cc84-43a1-a07c-0fe32b55f9bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: f7ffe8c2-a406-42d2-b8ba-89bf43352f83] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:27.237233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a9aa1e8b-cc84-43a1-a07c-0fe32b55f9bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:27.237798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a9aa1e8b-cc84-43a1-a07c-0fe32b55f9bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:27.238265 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a9aa1e8b-cc84-43a1-a07c-0fe32b55f9bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:27.238959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a9aa1e8b-cc84-43a1-a07c-0fe32b55f9bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:27.239418 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a9aa1e8b-cc84-43a1-a07c-0fe32b55f9bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:27.240055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-a9aa1e8b-cc84-43a1-a07c-0fe32b55f9bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:27.253150 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-a9aa1e8b-cc84-43a1-a07c-0fe32b55f9bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] HTTP exception thrown: Instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 could not be found. Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a9aa1e8b-cc84-43a1-a07c-0fe32b55f9bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Request method failure captured: Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 HTTP/1.1 Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfx-IBlTeU8lpWwGKB_djClew3IfsnnMsqAlTIyJwZatK6rO97X1sIwHrl836QwkkBwsDAti13QWgcktV2H3C3H2iyUMQVmkZ2rvFbwK_qsxWVKARvFf1vGfBVG_NZnxLXL77ZYDqLGjePARIcvrYdLR_yhLC_R9BR-JLVq-P_eoaHc Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: b21f666643b74b5cba0b7597ae1aa0cd Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeTestJSON-790394194 Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:35:27.254875 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:35:27.256280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:35:27.256280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeTestJSON-790394194 Sep 08 14:35:27.256280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: b21f666643b74b5cba0b7597ae1aa0cd Sep 08 14:35:27.256280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeTestJSON-790394194 Sep 08 14:35:27.256280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeTestJSON-790394194-project Sep 08 14:35:27.256280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:35:27.256280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:35:27.256280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 357ebb86fd664e08b15bf440e2a61560 Sep 08 14:35:27.256280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeTestJSON-790394194-project Sep 08 14:35:27.256280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:35:27.256280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 could not be found. Sep 08 14:35:27.256280 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'f7ffe8c2-a406-42d2-b8ba-89bf43352f83'} Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 could not be found. Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:35:27.257006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:35:27.258382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:35:27.258382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:35:27.258382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:35:27.258382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:35:27.258382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:35:27.258382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:35:27.258382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:35:27.258382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 could not be found. Sep 08 14:35:27.258382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:35:27.258382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:35:27.258382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:35:27.258382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:35:27.258382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:35:27.258382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:35:27.258382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:35:27.258382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:35:27.258382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 could not be found. Sep 08 14:35:27.258382 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:35:27.259542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-a9aa1e8b-cc84-43a1-a07c-0fe32b55f9bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Returning 404 to user: Instance f7ffe8c2-a406-42d2-b8ba-89bf43352f83 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:35:27.259542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-a9aa1e8b-cc84-43a1-a07c-0fe32b55f9bd tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83" status: 404 len: 111 microversion: 2.1 time: 0.030904 Sep 08 14:35:27.259811 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1566/2849] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:35:27 2022] GET /compute/v2.1/servers/f7ffe8c2-a406-42d2-b8ba-89bf43352f83 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:35:28.527814 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-438e9107-1250-4fd0-ae58-427ee51151b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:28.528309 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-438e9107-1250-4fd0-ae58-427ee51151b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:28.534774 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-438e9107-1250-4fd0-ae58-427ee51151b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:28.535258 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-438e9107-1250-4fd0-ae58-427ee51151b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:28.535703 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-438e9107-1250-4fd0-ae58-427ee51151b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:28.536391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-438e9107-1250-4fd0-ae58-427ee51151b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:28.536843 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-438e9107-1250-4fd0-ae58-427ee51151b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:28.537806 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-438e9107-1250-4fd0-ae58-427ee51151b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:28.552838 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-438e9107-1250-4fd0-ae58-427ee51151b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] HTTP exception thrown: Instance c09b905a-0795-4dc5-a413-007f3951e49f could not be found. Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-438e9107-1250-4fd0-ae58-427ee51151b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Request method failure captured: Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f HTTP/1.1 Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfx-IBlTeU8lpWwGKB_djClew3IfsnnMsqAlTIyJwZatK6rO97X1sIwHrl836QwkkBwsDAti13QWgcktV2H3C3H2iyUMQVmkZ2rvFbwK_qsxWVKARvFf1vGfBVG_NZnxLXL77ZYDqLGjePARIcvrYdLR_yhLC_R9BR-JLVq-P_eoaHc Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: b21f666643b74b5cba0b7597ae1aa0cd Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeTestJSON-790394194 Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:35:28.554434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:35:28.555742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:35:28.555742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeTestJSON-790394194 Sep 08 14:35:28.555742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: b21f666643b74b5cba0b7597ae1aa0cd Sep 08 14:35:28.555742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeTestJSON-790394194 Sep 08 14:35:28.555742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeTestJSON-790394194-project Sep 08 14:35:28.555742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:35:28.555742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:35:28.555742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 357ebb86fd664e08b15bf440e2a61560 Sep 08 14:35:28.555742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeTestJSON-790394194-project Sep 08 14:35:28.555742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:35:28.555742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance c09b905a-0795-4dc5-a413-007f3951e49f could not be found. Sep 08 14:35:28.555742 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'c09b905a-0795-4dc5-a413-007f3951e49f'} Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c09b905a-0795-4dc5-a413-007f3951e49f could not be found. Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:35:28.556387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c09b905a-0795-4dc5-a413-007f3951e49f could not be found. Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c09b905a-0795-4dc5-a413-007f3951e49f could not be found. Sep 08 14:35:28.557710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:35:28.558933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-438e9107-1250-4fd0-ae58-427ee51151b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Returning 404 to user: Instance c09b905a-0795-4dc5-a413-007f3951e49f could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:35:28.558933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-438e9107-1250-4fd0-ae58-427ee51151b9 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 404 len: 111 microversion: 2.1 time: 0.031957 Sep 08 14:35:28.558933 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1284/2850] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:35:28 2022] DELETE /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:35:28.564922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-81588165-171f-44c4-9232-c991d0e72e19 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:35:28.565537 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-81588165-171f-44c4-9232-c991d0e72e19 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] [instance: c09b905a-0795-4dc5-a413-007f3951e49f] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:35:28.572111 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81588165-171f-44c4-9232-c991d0e72e19 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:28.572598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81588165-171f-44c4-9232-c991d0e72e19 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:28.573036 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81588165-171f-44c4-9232-c991d0e72e19 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:28.573733 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81588165-171f-44c4-9232-c991d0e72e19 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:35:28.574224 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81588165-171f-44c4-9232-c991d0e72e19 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:35:28.574658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-81588165-171f-44c4-9232-c991d0e72e19 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:35:28.587902 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-81588165-171f-44c4-9232-c991d0e72e19 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] HTTP exception thrown: Instance c09b905a-0795-4dc5-a413-007f3951e49f could not be found. Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-81588165-171f-44c4-9232-c991d0e72e19 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Request method failure captured: Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f HTTP/1.1 Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfx-IBlTeU8lpWwGKB_djClew3IfsnnMsqAlTIyJwZatK6rO97X1sIwHrl836QwkkBwsDAti13QWgcktV2H3C3H2iyUMQVmkZ2rvFbwK_qsxWVKARvFf1vGfBVG_NZnxLXL77ZYDqLGjePARIcvrYdLR_yhLC_R9BR-JLVq-P_eoaHc Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: b21f666643b74b5cba0b7597ae1aa0cd Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeTestJSON-790394194 Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:35:28.589420 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:35:28.590828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/b21f666643b74b5cba0b7597ae1aa0cd"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:35:28.590828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeTestJSON-790394194 Sep 08 14:35:28.590828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: b21f666643b74b5cba0b7597ae1aa0cd Sep 08 14:35:28.590828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeTestJSON-790394194 Sep 08 14:35:28.590828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeTestJSON-790394194-project Sep 08 14:35:28.590828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:35:28.590828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:35:28.590828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 357ebb86fd664e08b15bf440e2a61560 Sep 08 14:35:28.590828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeTestJSON-790394194-project Sep 08 14:35:28.590828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:35:28.590828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance c09b905a-0795-4dc5-a413-007f3951e49f could not be found. Sep 08 14:35:28.590828 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'c09b905a-0795-4dc5-a413-007f3951e49f'} Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c09b905a-0795-4dc5-a413-007f3951e49f could not be found. Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:35:28.591546 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:35:28.592564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:35:28.592564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:35:28.592564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:35:28.592564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:35:28.592564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:35:28.592564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:35:28.592564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:35:28.592564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c09b905a-0795-4dc5-a413-007f3951e49f could not be found. Sep 08 14:35:28.592564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:35:28.592564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:35:28.592564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:35:28.592564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:35:28.592564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:35:28.592564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:35:28.592564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:35:28.592564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:35:28.592564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c09b905a-0795-4dc5-a413-007f3951e49f could not be found. Sep 08 14:35:28.592564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:35:28.593409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-81588165-171f-44c4-9232-c991d0e72e19 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] Returning 404 to user: Instance c09b905a-0795-4dc5-a413-007f3951e49f could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:35:28.593409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-81588165-171f-44c4-9232-c991d0e72e19 tempest-AttachVolumeTestJSON-790394194 tempest-AttachVolumeTestJSON-790394194-project] 10.222.0.25 "GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f" status: 404 len: 111 microversion: 2.1 time: 0.030289 Sep 08 14:35:28.593409 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1567/2851] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:35:28 2022] GET /compute/v2.1/servers/c09b905a-0795-4dc5-a413-007f3951e49f => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:36:29.084626 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=20 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:29.085365 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=50,nova_cell1:INSERT=2 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:29.086567 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Sep 08 14:36:29.086750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:36:29 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers (ip 10.222.0.25) !!! Sep 08 14:36:29.091494 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f1655141-0f2b-4651-bd56-dd7ffbb8eb29 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-689428700", "imageRef": "20b30425-92e5-4d78-8502-45557526ea49", "flavorRef": "452"}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:36:29.145448 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.network.neutron [None req-f1655141-0f2b-4651-bd56-dd7ffbb8eb29 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] validate_networks() for None {{(pid=127602) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 14:36:29.677941 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-f1655141-0f2b-4651-bd56-dd7ffbb8eb29 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Flavor limits 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 14:36:29.678386 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-f1655141-0f2b-4651-bd56-dd7ffbb8eb29 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Image limits 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 14:36:29.678847 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-f1655141-0f2b-4651-bd56-dd7ffbb8eb29 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Flavor pref 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 14:36:29.679249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-f1655141-0f2b-4651-bd56-dd7ffbb8eb29 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Image pref 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 14:36:29.679659 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-f1655141-0f2b-4651-bd56-dd7ffbb8eb29 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 14:36:29.691074 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f1655141-0f2b-4651-bd56-dd7ffbb8eb29 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:29.691543 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f1655141-0f2b-4651-bd56-dd7ffbb8eb29 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:29.691982 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f1655141-0f2b-4651-bd56-dd7ffbb8eb29 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:29.692689 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f1655141-0f2b-4651-bd56-dd7ffbb8eb29 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:29.693126 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f1655141-0f2b-4651-bd56-dd7ffbb8eb29 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:29.693540 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f1655141-0f2b-4651-bd56-dd7ffbb8eb29 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:29.711567 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-f1655141-0f2b-4651-bd56-dd7ffbb8eb29 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Getting quotas for project c690384cd06642628d5764761dc50d67. Resources: {'cores', 'instances', 'ram'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 14:36:29.715485 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-f1655141-0f2b-4651-bd56-dd7ffbb8eb29 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Getting quotas for user a4649b5cfb454ea1a2fadad4cda17995 and project c690384cd06642628d5764761dc50d67. Resources: {'cores', 'instances', 'ram'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 14:36:29.727165 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f1655141-0f2b-4651-bd56-dd7ffbb8eb29 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Going to run 1 instances... {{(pid=127602) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 14:36:29.729434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f1655141-0f2b-4651-bd56-dd7ffbb8eb29 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='20b30425-92e5-4d78-8502-45557526ea49',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127602) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 14:36:29.861947 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f1655141-0f2b-4651-bd56-dd7ffbb8eb29 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.779325 Sep 08 14:36:29.862817 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1285/2852] 10.222.0.25 () {60 vars in 1147 bytes} [Thu Sep 8 14:36:29 2022] POST /compute/v2.1/servers => generated 380 bytes in 781 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 08 14:36:29.872496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:36:29.875052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:36:29 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e (ip 10.222.0.25) !!! Sep 08 14:36:29.875052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=34,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:29.875052 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=77,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:29.879660 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-da1cfc26-e9e9-4320-8295-40e862ece8f0 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:29.880274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-da1cfc26-e9e9-4320-8295-40e862ece8f0 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:29.938310 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-da1cfc26-e9e9-4320-8295-40e862ece8f0 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:29.947295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-da1cfc26-e9e9-4320-8295-40e862ece8f0 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1214 microversion: 2.1 time: 0.076694 Sep 08 14:36:29.947862 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1568/2853] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:29 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1214 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:36:30.971178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2dcb17e7-b8f3-49e6-86b4-6b0845c718d0 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:30.971935 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2dcb17e7-b8f3-49e6-86b4-6b0845c718d0 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:31.030521 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-2dcb17e7-b8f3-49e6-86b4-6b0845c718d0 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:31.037455 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2dcb17e7-b8f3-49e6-86b4-6b0845c718d0 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1214 microversion: 2.1 time: 0.070356 Sep 08 14:36:31.038640 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1286/2854] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:30 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1214 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:36:32.061415 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d8a5e5c0-c85a-4988-a2ab-147447015244 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:32.062002 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d8a5e5c0-c85a-4988-a2ab-147447015244 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:32.117603 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d8a5e5c0-c85a-4988-a2ab-147447015244 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:32.124562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d8a5e5c0-c85a-4988-a2ab-147447015244 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1214 microversion: 2.1 time: 0.067731 Sep 08 14:36:32.125177 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1569/2855] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:32 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1214 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:36:33.147566 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2605bd26-54d9-43f4-b29e-1532eecd578c tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:33.148438 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2605bd26-54d9-43f4-b29e-1532eecd578c tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:33.201709 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-2605bd26-54d9-43f4-b29e-1532eecd578c tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:33.208582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2605bd26-54d9-43f4-b29e-1532eecd578c tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1214 microversion: 2.1 time: 0.064929 Sep 08 14:36:33.209137 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1287/2856] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:33 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1214 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:36:34.231777 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-3b6917b9-5d40-4139-9ed2-695904bba368 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:34.232386 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-3b6917b9-5d40-4139-9ed2-695904bba368 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:34.240054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3b6917b9-5d40-4139-9ed2-695904bba368 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:34.240542 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3b6917b9-5d40-4139-9ed2-695904bba368 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:34.240983 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3b6917b9-5d40-4139-9ed2-695904bba368 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:34.241674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3b6917b9-5d40-4139-9ed2-695904bba368 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:34.242132 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3b6917b9-5d40-4139-9ed2-695904bba368 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:34.242584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-3b6917b9-5d40-4139-9ed2-695904bba368 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:34.307232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-3b6917b9-5d40-4139-9ed2-695904bba368 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:34.315056 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-3b6917b9-5d40-4139-9ed2-695904bba368 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1218 microversion: 2.1 time: 0.087228 Sep 08 14:36:34.315614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1570/2857] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:34 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1218 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:36:35.338572 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6a90e749-3312-4b52-a0d1-dc973c581c89 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:35.339173 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-6a90e749-3312-4b52-a0d1-dc973c581c89 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:35.345614 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a90e749-3312-4b52-a0d1-dc973c581c89 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:35.346089 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a90e749-3312-4b52-a0d1-dc973c581c89 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:35.346543 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a90e749-3312-4b52-a0d1-dc973c581c89 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:35.347219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a90e749-3312-4b52-a0d1-dc973c581c89 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:35.347663 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a90e749-3312-4b52-a0d1-dc973c581c89 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:35.348079 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-6a90e749-3312-4b52-a0d1-dc973c581c89 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:35.421759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-6a90e749-3312-4b52-a0d1-dc973c581c89 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:35.429473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6a90e749-3312-4b52-a0d1-dc973c581c89 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1218 microversion: 2.1 time: 0.095264 Sep 08 14:36:35.430033 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1288/2858] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:35 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1218 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:36:36.451399 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8fa908bb-c5ae-49aa-b869-af8edf93f8ac tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:36.452020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8fa908bb-c5ae-49aa-b869-af8edf93f8ac tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:36.458739 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fa908bb-c5ae-49aa-b869-af8edf93f8ac tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:36.459233 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fa908bb-c5ae-49aa-b869-af8edf93f8ac tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:36.459685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fa908bb-c5ae-49aa-b869-af8edf93f8ac tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:36.460507 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fa908bb-c5ae-49aa-b869-af8edf93f8ac tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:36.460863 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fa908bb-c5ae-49aa-b869-af8edf93f8ac tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:36.461296 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8fa908bb-c5ae-49aa-b869-af8edf93f8ac tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:36.567560 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8fa908bb-c5ae-49aa-b869-af8edf93f8ac tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:36.575335 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8fa908bb-c5ae-49aa-b869-af8edf93f8ac tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1284 microversion: 2.1 time: 0.127925 Sep 08 14:36:36.575919 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1571/2859] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:36 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1284 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:36:37.601577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-3f43a9db-7d83-4938-9c91-f83d7d2c0d40 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:37.602178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-3f43a9db-7d83-4938-9c91-f83d7d2c0d40 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:37.608944 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3f43a9db-7d83-4938-9c91-f83d7d2c0d40 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:37.609429 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3f43a9db-7d83-4938-9c91-f83d7d2c0d40 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:37.609861 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3f43a9db-7d83-4938-9c91-f83d7d2c0d40 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:37.610569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3f43a9db-7d83-4938-9c91-f83d7d2c0d40 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:37.611020 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3f43a9db-7d83-4938-9c91-f83d7d2c0d40 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:37.611439 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-3f43a9db-7d83-4938-9c91-f83d7d2c0d40 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:37.748170 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-3f43a9db-7d83-4938-9c91-f83d7d2c0d40 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:37.755958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-3f43a9db-7d83-4938-9c91-f83d7d2c0d40 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1314 microversion: 2.1 time: 0.159231 Sep 08 14:36:37.757080 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1289/2860] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:37 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1314 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:36:38.423723 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-9c8e536d-75a4-44ec-ba65-67715e570029 req-50308dd5-c431-4046-a442-dd3c4137b7c3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c4038a70-e476-4dbc-8260-9677ae06658e", "tag": "a2a5bc19-b6d9-42dc-a7dc-0d6ff4338c12"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:36:38.433571 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-9c8e536d-75a4-44ec-ba65-67715e570029 req-50308dd5-c431-4046-a442-dd3c4137b7c3 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:38.434106 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-9c8e536d-75a4-44ec-ba65-67715e570029 req-50308dd5-c431-4046-a442-dd3c4137b7c3 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:38.434645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-9c8e536d-75a4-44ec-ba65-67715e570029 req-50308dd5-c431-4046-a442-dd3c4137b7c3 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:38.445642 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-9c8e536d-75a4-44ec-ba65-67715e570029 req-50308dd5-c431-4046-a442-dd3c4137b7c3 service nova] Creating event network-changed:a2a5bc19-b6d9-42dc-a7dc-0d6ff4338c12 for instance c4038a70-e476-4dbc-8260-9677ae06658e on hv-cinder-85646 Sep 08 14:36:38.452115 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-9c8e536d-75a4-44ec-ba65-67715e570029 req-50308dd5-c431-4046-a442-dd3c4137b7c3 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032125 Sep 08 14:36:38.452781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1572/2861] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:36:38 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:36:38.784969 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-98c32d0d-8fdf-48db-971e-5ea7d96223cc tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:38.786062 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-98c32d0d-8fdf-48db-971e-5ea7d96223cc tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:38.796510 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-98c32d0d-8fdf-48db-971e-5ea7d96223cc tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:38.797331 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-98c32d0d-8fdf-48db-971e-5ea7d96223cc tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:38.798086 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-98c32d0d-8fdf-48db-971e-5ea7d96223cc tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:38.799279 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-98c32d0d-8fdf-48db-971e-5ea7d96223cc tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:38.800073 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-98c32d0d-8fdf-48db-971e-5ea7d96223cc tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:38.800803 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-98c32d0d-8fdf-48db-971e-5ea7d96223cc tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:38.933325 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-98c32d0d-8fdf-48db-971e-5ea7d96223cc tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:38.940990 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-98c32d0d-8fdf-48db-971e-5ea7d96223cc tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1314 microversion: 2.1 time: 0.163289 Sep 08 14:36:38.941566 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1290/2862] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:38 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1314 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:36:39.966646 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-9561440c-0c85-4277-ad74-468aac106300 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:39.967268 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-9561440c-0c85-4277-ad74-468aac106300 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:39.973984 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9561440c-0c85-4277-ad74-468aac106300 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:39.974498 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9561440c-0c85-4277-ad74-468aac106300 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:39.974939 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9561440c-0c85-4277-ad74-468aac106300 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:39.975634 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9561440c-0c85-4277-ad74-468aac106300 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:39.976091 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9561440c-0c85-4277-ad74-468aac106300 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:39.977080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-9561440c-0c85-4277-ad74-468aac106300 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:40.106188 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-9561440c-0c85-4277-ad74-468aac106300 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:40.114098 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-9561440c-0c85-4277-ad74-468aac106300 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1314 microversion: 2.1 time: 0.152364 Sep 08 14:36:40.114781 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1573/2863] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:39 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1314 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:36:41.137133 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-00f44239-1bf1-45b8-bd0b-b09e12804768 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:41.137844 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-00f44239-1bf1-45b8-bd0b-b09e12804768 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:41.139582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=2 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:41.144907 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00f44239-1bf1-45b8-bd0b-b09e12804768 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:41.148291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00f44239-1bf1-45b8-bd0b-b09e12804768 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:41.148291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00f44239-1bf1-45b8-bd0b-b09e12804768 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:41.148291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00f44239-1bf1-45b8-bd0b-b09e12804768 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:41.148291 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00f44239-1bf1-45b8-bd0b-b09e12804768 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:41.148731 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-00f44239-1bf1-45b8-bd0b-b09e12804768 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:41.282725 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-00f44239-1bf1-45b8-bd0b-b09e12804768 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:41.291167 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-00f44239-1bf1-45b8-bd0b-b09e12804768 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1314 microversion: 2.1 time: 0.158160 Sep 08 14:36:41.291741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1291/2864] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:41 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1314 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:36:42.313974 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-42505666-cdd6-45a3-9993-eb7a4098bd10 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:42.314823 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-42505666-cdd6-45a3-9993-eb7a4098bd10 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:42.317333 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=4 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:42.323045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-42505666-cdd6-45a3-9993-eb7a4098bd10 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:42.323593 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-42505666-cdd6-45a3-9993-eb7a4098bd10 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:42.323990 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-42505666-cdd6-45a3-9993-eb7a4098bd10 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:42.324713 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-42505666-cdd6-45a3-9993-eb7a4098bd10 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:42.325180 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-42505666-cdd6-45a3-9993-eb7a4098bd10 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:42.325610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-42505666-cdd6-45a3-9993-eb7a4098bd10 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:42.473045 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-42505666-cdd6-45a3-9993-eb7a4098bd10 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:42.481326 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-42505666-cdd6-45a3-9993-eb7a4098bd10 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1314 microversion: 2.1 time: 0.171257 Sep 08 14:36:42.487316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1574/2865] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:42 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1314 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:36:43.505577 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-71230524-aa8a-4b4d-b650-eb3dc17a00ae tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:43.506210 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-71230524-aa8a-4b4d-b650-eb3dc17a00ae tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:43.508493 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=10 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:36:43.513776 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71230524-aa8a-4b4d-b650-eb3dc17a00ae tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:43.514294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71230524-aa8a-4b4d-b650-eb3dc17a00ae tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:43.514728 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71230524-aa8a-4b4d-b650-eb3dc17a00ae tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:43.515412 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71230524-aa8a-4b4d-b650-eb3dc17a00ae tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:43.515914 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71230524-aa8a-4b4d-b650-eb3dc17a00ae tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:43.516331 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-71230524-aa8a-4b4d-b650-eb3dc17a00ae tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:43.653456 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-71230524-aa8a-4b4d-b650-eb3dc17a00ae tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:43.661161 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-71230524-aa8a-4b4d-b650-eb3dc17a00ae tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1314 microversion: 2.1 time: 0.159536 Sep 08 14:36:43.661731 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1292/2866] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:43 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1314 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:36:44.396951 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-e4aca425-6bfc-4edc-ae22-2e1af8cb1cf3 req-ccb58071-2857-4b9c-b599-4b72a1e759be service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c4038a70-e476-4dbc-8260-9677ae06658e", "name": "network-vif-plugged", "status": "completed", "tag": "a2a5bc19-b6d9-42dc-a7dc-0d6ff4338c12"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:36:44.410009 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-e4aca425-6bfc-4edc-ae22-2e1af8cb1cf3 req-ccb58071-2857-4b9c-b599-4b72a1e759be service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:44.410529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-e4aca425-6bfc-4edc-ae22-2e1af8cb1cf3 req-ccb58071-2857-4b9c-b599-4b72a1e759be service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:44.410966 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-e4aca425-6bfc-4edc-ae22-2e1af8cb1cf3 req-ccb58071-2857-4b9c-b599-4b72a1e759be service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:44.423026 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-e4aca425-6bfc-4edc-ae22-2e1af8cb1cf3 req-ccb58071-2857-4b9c-b599-4b72a1e759be service nova] Creating event network-vif-plugged:a2a5bc19-b6d9-42dc-a7dc-0d6ff4338c12 for instance c4038a70-e476-4dbc-8260-9677ae06658e on hv-cinder-85646 Sep 08 14:36:44.428197 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-e4aca425-6bfc-4edc-ae22-2e1af8cb1cf3 req-ccb58071-2857-4b9c-b599-4b72a1e759be service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.167061 Sep 08 14:36:44.428789 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1575/2867] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:36:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:36:44.685231 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-f81ab21b-10a2-4d40-aa05-3da17d8467e3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:44.685793 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-f81ab21b-10a2-4d40-aa05-3da17d8467e3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:44.692278 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f81ab21b-10a2-4d40-aa05-3da17d8467e3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:44.692754 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f81ab21b-10a2-4d40-aa05-3da17d8467e3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:44.693180 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f81ab21b-10a2-4d40-aa05-3da17d8467e3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:44.693857 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f81ab21b-10a2-4d40-aa05-3da17d8467e3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:44.694341 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f81ab21b-10a2-4d40-aa05-3da17d8467e3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:44.694759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-f81ab21b-10a2-4d40-aa05-3da17d8467e3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:44.827759 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-f81ab21b-10a2-4d40-aa05-3da17d8467e3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:44.835298 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-f81ab21b-10a2-4d40-aa05-3da17d8467e3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1314 microversion: 2.1 time: 0.155292 Sep 08 14:36:44.835875 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1293/2868] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:44 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1314 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:36:45.860104 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-acd84669-6f36-4c81-afb2-1560b1dc0cf2 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:45.860798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-acd84669-6f36-4c81-afb2-1560b1dc0cf2 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:45.867799 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-acd84669-6f36-4c81-afb2-1560b1dc0cf2 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:45.868278 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-acd84669-6f36-4c81-afb2-1560b1dc0cf2 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:45.868711 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-acd84669-6f36-4c81-afb2-1560b1dc0cf2 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:45.869401 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-acd84669-6f36-4c81-afb2-1560b1dc0cf2 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:45.869846 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-acd84669-6f36-4c81-afb2-1560b1dc0cf2 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:45.870314 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-acd84669-6f36-4c81-afb2-1560b1dc0cf2 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:46.002111 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-acd84669-6f36-4c81-afb2-1560b1dc0cf2 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:46.009853 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-acd84669-6f36-4c81-afb2-1560b1dc0cf2 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1314 microversion: 2.1 time: 0.154442 Sep 08 14:36:46.010445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1576/2869] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:45 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1314 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:36:47.033383 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-277dddb8-1f5a-4718-919d-d901a4877e36 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:47.033995 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-277dddb8-1f5a-4718-919d-d901a4877e36 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:47.041960 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-277dddb8-1f5a-4718-919d-d901a4877e36 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:47.042479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-277dddb8-1f5a-4718-919d-d901a4877e36 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:47.042917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-277dddb8-1f5a-4718-919d-d901a4877e36 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:47.043609 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-277dddb8-1f5a-4718-919d-d901a4877e36 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:47.044065 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-277dddb8-1f5a-4718-919d-d901a4877e36 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:47.044489 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-277dddb8-1f5a-4718-919d-d901a4877e36 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:47.180333 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-277dddb8-1f5a-4718-919d-d901a4877e36 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:47.188829 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-277dddb8-1f5a-4718-919d-d901a4877e36 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1463 microversion: 2.1 time: 0.159301 Sep 08 14:36:47.189398 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1294/2870] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:47 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1463 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:36:47.236532 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-01aa0b8f-8ca5-4c04-96f3-df5f5a0db6f7 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:47.237013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-01aa0b8f-8ca5-4c04-96f3-df5f5a0db6f7 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:47.243801 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01aa0b8f-8ca5-4c04-96f3-df5f5a0db6f7 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:47.244336 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01aa0b8f-8ca5-4c04-96f3-df5f5a0db6f7 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:47.245647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01aa0b8f-8ca5-4c04-96f3-df5f5a0db6f7 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:47.246100 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01aa0b8f-8ca5-4c04-96f3-df5f5a0db6f7 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:47.246584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01aa0b8f-8ca5-4c04-96f3-df5f5a0db6f7 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:47.247007 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-01aa0b8f-8ca5-4c04-96f3-df5f5a0db6f7 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:47.274614 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-01aa0b8f-8ca5-4c04-96f3-df5f5a0db6f7 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Going to try to terminate instance {{(pid=127601) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:36:47.550351 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-01aa0b8f-8ca5-4c04-96f3-df5f5a0db6f7 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 204 len: 0 microversion: 2.1 time: 0.317295 Sep 08 14:36:47.550939 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1577/2871] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:36:47 2022] DELETE /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 0 bytes in 319 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:36:47.560274 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-355af00f-514e-436f-baf2-f9d4b4514510 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:47.561010 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-355af00f-514e-436f-baf2-f9d4b4514510 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:47.567151 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-355af00f-514e-436f-baf2-f9d4b4514510 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:47.567644 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-355af00f-514e-436f-baf2-f9d4b4514510 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:47.568107 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-355af00f-514e-436f-baf2-f9d4b4514510 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:47.568864 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-355af00f-514e-436f-baf2-f9d4b4514510 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:47.569316 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-355af00f-514e-436f-baf2-f9d4b4514510 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:47.569740 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-355af00f-514e-436f-baf2-f9d4b4514510 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:47.710475 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-355af00f-514e-436f-baf2-f9d4b4514510 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:47.718616 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-355af00f-514e-436f-baf2-f9d4b4514510 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1469 microversion: 2.1 time: 0.163152 Sep 08 14:36:47.719205 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1295/2872] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:47 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1469 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:36:48.743303 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-e99ff157-16db-4ea2-b3c7-ca35e88b7aa8 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:48.744006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-e99ff157-16db-4ea2-b3c7-ca35e88b7aa8 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:48.751338 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e99ff157-16db-4ea2-b3c7-ca35e88b7aa8 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:48.751829 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e99ff157-16db-4ea2-b3c7-ca35e88b7aa8 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:48.752295 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e99ff157-16db-4ea2-b3c7-ca35e88b7aa8 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:48.752999 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e99ff157-16db-4ea2-b3c7-ca35e88b7aa8 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:48.753481 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e99ff157-16db-4ea2-b3c7-ca35e88b7aa8 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:48.753907 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-e99ff157-16db-4ea2-b3c7-ca35e88b7aa8 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:48.894723 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-e99ff157-16db-4ea2-b3c7-ca35e88b7aa8 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:48.903368 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-e99ff157-16db-4ea2-b3c7-ca35e88b7aa8 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1469 microversion: 2.1 time: 0.164277 Sep 08 14:36:48.903943 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1578/2873] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:48 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1469 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:36:49.746569 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-d904555e-8511-4a05-8041-2707c4de57fe req-90ee020f-6312-4082-9a58-d2705b07d7dc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c4038a70-e476-4dbc-8260-9677ae06658e", "name": "network-vif-deleted", "tag": "a2a5bc19-b6d9-42dc-a7dc-0d6ff4338c12"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:36:49.763075 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-d904555e-8511-4a05-8041-2707c4de57fe req-90ee020f-6312-4082-9a58-d2705b07d7dc service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:49.763595 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-d904555e-8511-4a05-8041-2707c4de57fe req-90ee020f-6312-4082-9a58-d2705b07d7dc service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:49.764026 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-d904555e-8511-4a05-8041-2707c4de57fe req-90ee020f-6312-4082-9a58-d2705b07d7dc service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:49.778032 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-d904555e-8511-4a05-8041-2707c4de57fe req-90ee020f-6312-4082-9a58-d2705b07d7dc service nova] Creating event network-vif-deleted:a2a5bc19-b6d9-42dc-a7dc-0d6ff4338c12 for instance c4038a70-e476-4dbc-8260-9677ae06658e on hv-cinder-85646 Sep 08 14:36:49.784699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-d904555e-8511-4a05-8041-2707c4de57fe req-90ee020f-6312-4082-9a58-d2705b07d7dc service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046045 Sep 08 14:36:49.784699 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1296/2874] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:36:49 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:36:49.926865 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6d856988-5dcf-478e-8eef-9d6fe42b84b3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:49.927469 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6d856988-5dcf-478e-8eef-9d6fe42b84b3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:49.934831 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6d856988-5dcf-478e-8eef-9d6fe42b84b3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:49.935313 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6d856988-5dcf-478e-8eef-9d6fe42b84b3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:49.935750 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6d856988-5dcf-478e-8eef-9d6fe42b84b3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:49.936441 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6d856988-5dcf-478e-8eef-9d6fe42b84b3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:49.936897 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6d856988-5dcf-478e-8eef-9d6fe42b84b3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:49.937324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6d856988-5dcf-478e-8eef-9d6fe42b84b3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:50.003932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6d856988-5dcf-478e-8eef-9d6fe42b84b3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'a4649b5cfb454ea1a2fadad4cda17995', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'c690384cd06642628d5764761dc50d67', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:36:50.012028 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6d856988-5dcf-478e-8eef-9d6fe42b84b3 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 200 len: 1299 microversion: 2.1 time: 0.089001 Sep 08 14:36:50.012604 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1579/2875] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:49 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 1299 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:36:51.036382 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7c95ebfd-49cc-4fd4-b648-b08580a70913 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:36:51.037008 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7c95ebfd-49cc-4fd4-b648-b08580a70913 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] [instance: c4038a70-e476-4dbc-8260-9677ae06658e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:36:51.043443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c95ebfd-49cc-4fd4-b648-b08580a70913 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:51.043936 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c95ebfd-49cc-4fd4-b648-b08580a70913 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:51.044385 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c95ebfd-49cc-4fd4-b648-b08580a70913 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:51.045106 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c95ebfd-49cc-4fd4-b648-b08580a70913 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:36:51.045561 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c95ebfd-49cc-4fd4-b648-b08580a70913 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:36:51.045986 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7c95ebfd-49cc-4fd4-b648-b08580a70913 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:36:51.061864 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-7c95ebfd-49cc-4fd4-b648-b08580a70913 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] HTTP exception thrown: Instance c4038a70-e476-4dbc-8260-9677ae06658e could not be found. Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7c95ebfd-49cc-4fd4-b648-b08580a70913 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Request method failure captured: Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e HTTP/1.1 Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGf3rGla5JHwczHl_DRw5NoQaGP9aL3FH3G6Lt0qi-adhWTz-mD96iCQGzbWNkSYkF33W9S8ZtnMTKJj2DAov5xAZmqWXS3FZhvhd7MBXrZVSiVXqBd2c1abICvGaAJ5bkpYlBtrgFi8xWASy5nJNDsHywBstmKYJKyxPoD2MyfqoYjI Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: c690384cd06642628d5764761dc50d67 Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-VolumesNegativeTest-76804668 Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:36:51.063562 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:36:51.064958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_c690384cd06642628d5764761dc50d67"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c690384cd06642628d5764761dc50d67"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/c690384cd06642628d5764761dc50d67"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/c690384cd06642628d5764761dc50d67"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:36:51.064958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-VolumesNegativeTest-76804668 Sep 08 14:36:51.064958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: c690384cd06642628d5764761dc50d67 Sep 08 14:36:51.064958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-VolumesNegativeTest-76804668 Sep 08 14:36:51.064958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-VolumesNegativeTest-76804668-project Sep 08 14:36:51.064958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:36:51.064958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:36:51.064958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: a4649b5cfb454ea1a2fadad4cda17995 Sep 08 14:36:51.064958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-VolumesNegativeTest-76804668-project Sep 08 14:36:51.064958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:36:51.064958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance c4038a70-e476-4dbc-8260-9677ae06658e could not be found. Sep 08 14:36:51.064958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'c4038a70-e476-4dbc-8260-9677ae06658e'} Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c4038a70-e476-4dbc-8260-9677ae06658e could not be found. Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:36:51.065931 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:36:51.067312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:36:51.067312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:36:51.067312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:36:51.067312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:36:51.067312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:36:51.067312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:36:51.067312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:36:51.067312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c4038a70-e476-4dbc-8260-9677ae06658e could not be found. Sep 08 14:36:51.067312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:36:51.067312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:36:51.067312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:36:51.067312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:36:51.067312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:36:51.067312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:36:51.067312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:36:51.067312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:36:51.067312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c4038a70-e476-4dbc-8260-9677ae06658e could not be found. Sep 08 14:36:51.067312 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:36:51.068652 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7c95ebfd-49cc-4fd4-b648-b08580a70913 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] Returning 404 to user: Instance c4038a70-e476-4dbc-8260-9677ae06658e could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:36:51.068652 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7c95ebfd-49cc-4fd4-b648-b08580a70913 tempest-VolumesNegativeTest-76804668 tempest-VolumesNegativeTest-76804668-project] 10.222.0.25 "GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e" status: 404 len: 111 microversion: 2.1 time: 0.033827 Sep 08 14:36:51.068652 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1297/2876] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:36:51 2022] GET /compute/v2.1/servers/c4038a70-e476-4dbc-8260-9677ae06658e => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:39:05.308170 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-49292a2f-c7ad-43b2-ac2f-ae17f26e834d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:39:05.308749 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-49292a2f-c7ad-43b2-ac2f-ae17f26e834d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:39:05.311129 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=33,nova_api:UPDATE=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:05.316945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=60,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:05.316945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:39:05.316945 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Thu Sep 8 14:39:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 (ip 10.222.0.25) !!! Sep 08 14:39:05.321989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-49292a2f-c7ad-43b2-ac2f-ae17f26e834d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:39:05.322503 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-49292a2f-c7ad-43b2-ac2f-ae17f26e834d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:39:05.322940 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-49292a2f-c7ad-43b2-ac2f-ae17f26e834d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:39:05.323633 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-49292a2f-c7ad-43b2-ac2f-ae17f26e834d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:39:05.324083 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-49292a2f-c7ad-43b2-ac2f-ae17f26e834d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:39:05.324541 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-49292a2f-c7ad-43b2-ac2f-ae17f26e834d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:39:05.339718 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-49292a2f-c7ad-43b2-ac2f-ae17f26e834d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] HTTP exception thrown: Instance c14f6817-e3ab-4a36-ba23-590498bf1b25 could not be found. Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-49292a2f-c7ad-43b2-ac2f-ae17f26e834d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Request method failure captured: Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 HTTP/1.1 Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGfPpgseYqW3vfBhChnnflETRt0GDtjL3VmhITeNLuRSALZefRpZ4LlXgFeo5cygwUtHgGuuBrrkR39Uo-ZWGIlwSJVssX0SoJbjjdLhYxjZOutCiSryd_0UQo0F3jz05Y7DAMd5jNO7kUKOE8WKEghkWPfZegSJZuLFAO1D7uF_weRo Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Openstack-Nova-Api-Version: 2.20 Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:39:05.341424 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:39:05.342922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:39:05.342922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:39:05.342922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:39:05.342922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:39:05.342922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:39:05.342922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:39:05.342922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:39:05.342922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: e9925313d1d446fa81a00cb55348dff7 Sep 08 14:39:05.342922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:39:05.342922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:39:05.342922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance c14f6817-e3ab-4a36-ba23-590498bf1b25 could not be found. Sep 08 14:39:05.342922 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'c14f6817-e3ab-4a36-ba23-590498bf1b25'} Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c14f6817-e3ab-4a36-ba23-590498bf1b25 could not be found. Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:39:05.343890 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c14f6817-e3ab-4a36-ba23-590498bf1b25 could not be found. Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c14f6817-e3ab-4a36-ba23-590498bf1b25 could not be found. Sep 08 14:39:05.345215 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:05.346742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-49292a2f-c7ad-43b2-ac2f-ae17f26e834d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Returning 404 to user: Instance c14f6817-e3ab-4a36-ba23-590498bf1b25 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:39:05.346742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-49292a2f-c7ad-43b2-ac2f-ae17f26e834d tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 404 len: 111 microversion: 2.20 time: 0.041253 Sep 08 14:39:05.346742 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1580/2877] 10.222.0.25 () {60 vars in 1280 bytes} [Thu Sep 8 14:39:05 2022] DELETE /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 08 14:39:05.350299 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR oslo.messaging._drivers.impl_rabbit [-] [9540fb82-3e3d-4223-b063-5eea119fb7c0] AMQP server on 10.222.0.25:5672 is unreachable: . Trying again in 1 seconds.: amqp.exceptions.RecoverableConnectionError: Sep 08 14:39:05.351104 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=41,nova_api:INSERT=3 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:05.351813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=54 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:05.352584 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 08 14:39:05.352719 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Thu Sep 8 14:39:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 (ip 10.222.0.25) !!! Sep 08 14:39:05.357881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-44ca8516-66b7-4530-8fc6-ca727e1332bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:39:05.358503 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-44ca8516-66b7-4530-8fc6-ca727e1332bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] [instance: c14f6817-e3ab-4a36-ba23-590498bf1b25] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:39:05.365403 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-44ca8516-66b7-4530-8fc6-ca727e1332bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:39:05.365878 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-44ca8516-66b7-4530-8fc6-ca727e1332bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:39:05.366888 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-44ca8516-66b7-4530-8fc6-ca727e1332bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:39:05.367571 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-44ca8516-66b7-4530-8fc6-ca727e1332bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:39:05.368014 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-44ca8516-66b7-4530-8fc6-ca727e1332bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:39:05.368430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-44ca8516-66b7-4530-8fc6-ca727e1332bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:39:05.383444 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-44ca8516-66b7-4530-8fc6-ca727e1332bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] HTTP exception thrown: Instance c14f6817-e3ab-4a36-ba23-590498bf1b25 could not be found. Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-44ca8516-66b7-4530-8fc6-ca727e1332bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Request method failure captured: Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 HTTP/1.1 Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGfPpgseYqW3vfBhChnnflETRt0GDtjL3VmhITeNLuRSALZefRpZ4LlXgFeo5cygwUtHgGuuBrrkR39Uo-ZWGIlwSJVssX0SoJbjjdLhYxjZOutCiSryd_0UQo0F3jz05Y7DAMd5jNO7kUKOE8WKEghkWPfZegSJZuLFAO1D7uF_weRo Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Openstack-Nova-Api-Version: 2.20 Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:39:05.384958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:39:05.387340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/4c194dd986d442d3bcaa83442a49ff70"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:39:05.387340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:39:05.387340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 4c194dd986d442d3bcaa83442a49ff70 Sep 08 14:39:05.387340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-390653242 Sep 08 14:39:05.387340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:39:05.387340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:39:05.387340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:39:05.387340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: e9925313d1d446fa81a00cb55348dff7 Sep 08 14:39:05.387340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-AttachVolumeShelveTestJSON-390653242-project Sep 08 14:39:05.387340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: > Sep 08 14:39:05.387340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Instance c14f6817-e3ab-4a36-ba23-590498bf1b25 could not be found. Sep 08 14:39:05.387340 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'c14f6817-e3ab-4a36-ba23-590498bf1b25'} Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c14f6817-e3ab-4a36-ba23-590498bf1b25 could not be found. Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:39:05.388785 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:39:05.390741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:39:05.390741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:39:05.390741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:39:05.390741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:39:05.390741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:39:05.390741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:39:05.390741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:39:05.390741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c14f6817-e3ab-4a36-ba23-590498bf1b25 could not be found. Sep 08 14:39:05.390741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:05.390741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:05.390741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:05.390741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:05.390741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:39:05.390741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:39:05.390741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:39:05.390741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:39:05.390741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c14f6817-e3ab-4a36-ba23-590498bf1b25 could not be found. Sep 08 14:39:05.390741 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:05.392714 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-44ca8516-66b7-4530-8fc6-ca727e1332bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] Returning 404 to user: Instance c14f6817-e3ab-4a36-ba23-590498bf1b25 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:39:05.392714 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-44ca8516-66b7-4530-8fc6-ca727e1332bc tempest-AttachVolumeShelveTestJSON-390653242 tempest-AttachVolumeShelveTestJSON-390653242-project] 10.222.0.25 "GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25" status: 404 len: 111 microversion: 2.20 time: 0.038625 Sep 08 14:39:05.392714 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1298/2878] 10.222.0.25 () {60 vars in 1277 bytes} [Thu Sep 8 14:39:05 2022] GET /compute/v2.1/servers/c14f6817-e3ab-4a36-ba23-590498bf1b25 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 08 14:39:10.942823 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ec2c887f-b5e5-4785-a898-4a220b65e405 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] Action: 'create', calling method: .version_select at 0x7fce0262ec10>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1622454242", "metadata": {"Type": "work"}}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ec2c887f-b5e5-4785-a898-4a220b65e405 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] Request method failure captured: Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 122 Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGf6OgdhQ_WRbJnJZ6wWxULuFHUcuMdscj7Jm7BR7IW1gM0fhHw62oo6kdOnfhyRJwbRIO6TuF8MTZK3l2fSB4M3uY5nP-syNijY8ekMcuns83Lq0TVrhvWPSiM2SV9BRpVD0Wz_fvaTH6kurL0dQe6KWWKBN7UuDVWVKVX-RfD4yZYY Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 2c05b7c0d85a4a04a64aa3f9b25aa9bf Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-VolumesNegativeTest-1043166854 Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:39:10.959764 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:39:10.961329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:39:10.961329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-VolumesNegativeTest-1043166854 Sep 08 14:39:10.961329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 2c05b7c0d85a4a04a64aa3f9b25aa9bf Sep 08 14:39:10.961329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-VolumesNegativeTest-1043166854 Sep 08 14:39:10.961329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-VolumesNegativeTest-1043166854-project Sep 08 14:39:10.961329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:39:10.961329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:39:10.961329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 437982b7ed6d47dd90e0232740445e69 Sep 08 14:39:10.961329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-VolumesNegativeTest-1043166854-project Sep 08 14:39:10.961329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Sep 08 14:39:10.961329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1622454242", "metadata": {"Type": "work"}}} Sep 08 14:39:10.961329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: Sep 08 14:39:10.961329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1622454242', 'metadata': {'Type': 'work'}}}} Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise error Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi '#$%' Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:39:10.962316 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 08 14:39:10.963821 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 08 14:39:10.965126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 08 14:39:10.965126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.965126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ec2c887f-b5e5-4785-a898-4a220b65e405 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:39:10.965126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ec2c887f-b5e5-4785-a898-4a220b65e405 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.133787 Sep 08 14:39:10.965126 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1581/2879] 10.222.0.25 () {60 vars in 1156 bytes} [Thu Sep 8 14:39:10 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 135 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 08 14:39:10.970959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fdd0279e-d2c4-4d92-80d9-6d27282a90e6 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] Action: 'create', calling method: .version_select at 0x7fce059a0160>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1413124001", "metadata": {"Type": "work"}}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fdd0279e-d2c4-4d92-80d9-6d27282a90e6 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] Request method failure captured: Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 120 Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGf6OgdhQ_WRbJnJZ6wWxULuFHUcuMdscj7Jm7BR7IW1gM0fhHw62oo6kdOnfhyRJwbRIO6TuF8MTZK3l2fSB4M3uY5nP-syNijY8ekMcuns83Lq0TVrhvWPSiM2SV9BRpVD0Wz_fvaTH6kurL0dQe6KWWKBN7UuDVWVKVX-RfD4yZYY Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 2c05b7c0d85a4a04a64aa3f9b25aa9bf Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-VolumesNegativeTest-1043166854 Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:39:10.975473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:39:10.976901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:39:10.976901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-VolumesNegativeTest-1043166854 Sep 08 14:39:10.976901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 2c05b7c0d85a4a04a64aa3f9b25aa9bf Sep 08 14:39:10.976901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-VolumesNegativeTest-1043166854 Sep 08 14:39:10.976901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-VolumesNegativeTest-1043166854-project Sep 08 14:39:10.976901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:39:10.976901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:39:10.976901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 437982b7ed6d47dd90e0232740445e69 Sep 08 14:39:10.976901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-VolumesNegativeTest-1043166854-project Sep 08 14:39:10.976901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Sep 08 14:39:10.976901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-1413124001", "metadata": {"Type": "work"}}} Sep 08 14:39:10.976901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 14:39:10.976901 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-1413124001', 'metadata': {'Type': 'work'}}}} Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise error Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi '0' Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:39:10.977713 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 08 14:39:10.979168 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 08 14:39:10.980353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 08 14:39:10.980353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.980353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-fdd0279e-d2c4-4d92-80d9-6d27282a90e6 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:39:10.980353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-fdd0279e-d2c4-4d92-80d9-6d27282a90e6 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.009812 Sep 08 14:39:10.980353 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1299/2880] 10.222.0.25 () {60 vars in 1156 bytes} [Thu Sep 8 14:39:10 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 11 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 08 14:39:10.987192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-76e4c882-e036-400a-b348-95cd4fd2b777 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] Action: 'create', calling method: .version_select at 0x7fce059461f0>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1750787814", "metadata": {"Type": "work"}}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-76e4c882-e036-400a-b348-95cd4fd2b777 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] Request method failure captured: Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 119 Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGf6OgdhQ_WRbJnJZ6wWxULuFHUcuMdscj7Jm7BR7IW1gM0fhHw62oo6kdOnfhyRJwbRIO6TuF8MTZK3l2fSB4M3uY5nP-syNijY8ekMcuns83Lq0TVrhvWPSiM2SV9BRpVD0Wz_fvaTH6kurL0dQe6KWWKBN7UuDVWVKVX-RfD4yZYY Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 2c05b7c0d85a4a04a64aa3f9b25aa9bf Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-VolumesNegativeTest-1043166854 Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:39:10.992432 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:39:10.993903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:39:10.993903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-VolumesNegativeTest-1043166854 Sep 08 14:39:10.993903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 2c05b7c0d85a4a04a64aa3f9b25aa9bf Sep 08 14:39:10.993903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-VolumesNegativeTest-1043166854 Sep 08 14:39:10.993903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-VolumesNegativeTest-1043166854-project Sep 08 14:39:10.993903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:39:10.993903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:39:10.993903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 437982b7ed6d47dd90e0232740445e69 Sep 08 14:39:10.993903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-VolumesNegativeTest-1043166854-project Sep 08 14:39:10.993903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Sep 08 14:39:10.993903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-1750787814", "metadata": {"Type": "work"}}} Sep 08 14:39:10.993903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: Sep 08 14:39:10.993903 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-1750787814', 'metadata': {'Type': 'work'}}}} Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise error Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi '' Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:39:10.994959 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 08 14:39:10.996483 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 08 14:39:10.997717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 08 14:39:10.997717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:10.997717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-76e4c882-e036-400a-b348-95cd4fd2b777 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:39:10.997717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-76e4c882-e036-400a-b348-95cd4fd2b777 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] 10.222.0.25 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.010837 Sep 08 14:39:10.997717 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1582/2881] 10.222.0.25 () {60 vars in 1156 bytes} [Thu Sep 8 14:39:10 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 12 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 08 14:39:11.011160 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-40896818-8d21-437d-b142-9fa1ab054589 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] Calling method '.version_select at 0x7fce02181790>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:39:11.025387 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [-] [9540fb82-3e3d-4223-b063-5eea119fb7c0] Reconnected to AMQP server on 10.222.0.25:5672 via [amqp] client with port 44310. Sep 08 14:39:11.144121 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-40896818-8d21-437d-b142-9fa1ab054589 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] HTTP exception thrown: Volume tempest-invalid-599726850 could not be found. Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-40896818-8d21-437d-b142-9fa1ab054589 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] Request method failure captured: Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-599726850 HTTP/1.1 Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGf6OgdhQ_WRbJnJZ6wWxULuFHUcuMdscj7Jm7BR7IW1gM0fhHw62oo6kdOnfhyRJwbRIO6TuF8MTZK3l2fSB4M3uY5nP-syNijY8ekMcuns83Lq0TVrhvWPSiM2SV9BRpVD0Wz_fvaTH6kurL0dQe6KWWKBN7UuDVWVKVX-RfD4yZYY Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 2c05b7c0d85a4a04a64aa3f9b25aa9bf Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-VolumesNegativeTest-1043166854 Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:39:11.146434 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:39:11.147813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:39:11.147813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-VolumesNegativeTest-1043166854 Sep 08 14:39:11.147813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 2c05b7c0d85a4a04a64aa3f9b25aa9bf Sep 08 14:39:11.147813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-VolumesNegativeTest-1043166854 Sep 08 14:39:11.147813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-VolumesNegativeTest-1043166854-project Sep 08 14:39:11.147813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:39:11.147813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:39:11.147813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 437982b7ed6d47dd90e0232740445e69 Sep 08 14:39:11.147813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-VolumesNegativeTest-1043166854-project Sep 08 14:39:11.147813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 14:39:11.147813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Volume tempest-invalid-599726850 could not be found. Sep 08 14:39:11.147813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': 'tempest-invalid-599726850'} Sep 08 14:39:11.147813 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-599726850 could not be found. (HTTP 404) (Request-ID: req-10bcde58-46ec-40ea-a43f-ebd8b649d6cb) Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:11.148746 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 08 14:39:11.150099 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-599726850 could not be found. Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:39:11.151394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-599726850 could not be found. Sep 08 14:39:11.153012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.153012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:11.153012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.153012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:11.153012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:39:11.153012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:39:11.153012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:39:11.153012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:39:11.153012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-599726850 could not be found. Sep 08 14:39:11.153012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.153012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-40896818-8d21-437d-b142-9fa1ab054589 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] Returning 404 to user: Volume tempest-invalid-599726850 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:39:11.153012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-40896818-8d21-437d-b142-9fa1ab054589 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/tempest-invalid-599726850" status: 404 len: 98 microversion: 2.1 time: 0.142888 Sep 08 14:39:11.153012 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1300/2882] 10.222.0.25 () {58 vars in 1215 bytes} [Thu Sep 8 14:39:11 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-599726850 => generated 98 bytes in 144 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 08 14:39:11.158168 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-960ce848-f3c3-43fc-8d67-3c40e1186723 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004056 Sep 08 14:39:11.158667 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1583/2883] 10.222.0.25 () {58 vars in 1140 bytes} [Thu Sep 8 14:39:11 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 08 14:39:11.166886 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-861a0d3b-638a-4323-b9e2-3f5c6ee6f17f tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003752 Sep 08 14:39:11.167347 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1301/2884] 10.222.0.25 () {58 vars in 1137 bytes} [Thu Sep 8 14:39:11 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 08 14:39:11.175981 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-422644ef-3e51-452c-b5c6-1c09f09104cc tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] Calling method '.version_select at 0x7fce02f2b310>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:39:11.258039 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-422644ef-3e51-452c-b5c6-1c09f09104cc tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] HTTP exception thrown: Volume 5801a83f-0f7d-4357-b436-6cd8e4ea7a35 could not be found. Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-422644ef-3e51-452c-b5c6-1c09f09104cc tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] Request method failure captured: Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: DELETE /compute/v2.1/os-volumes/5801a83f-0f7d-4357-b436-6cd8e4ea7a35 HTTP/1.1 Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGf6OgdhQ_WRbJnJZ6wWxULuFHUcuMdscj7Jm7BR7IW1gM0fhHw62oo6kdOnfhyRJwbRIO6TuF8MTZK3l2fSB4M3uY5nP-syNijY8ekMcuns83Lq0TVrhvWPSiM2SV9BRpVD0Wz_fvaTH6kurL0dQe6KWWKBN7UuDVWVKVX-RfD4yZYY Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 2c05b7c0d85a4a04a64aa3f9b25aa9bf Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-VolumesNegativeTest-1043166854 Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:39:11.260054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:39:11.261496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:39:11.261496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-VolumesNegativeTest-1043166854 Sep 08 14:39:11.261496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 2c05b7c0d85a4a04a64aa3f9b25aa9bf Sep 08 14:39:11.261496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-VolumesNegativeTest-1043166854 Sep 08 14:39:11.261496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-VolumesNegativeTest-1043166854-project Sep 08 14:39:11.261496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:39:11.261496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:39:11.261496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 437982b7ed6d47dd90e0232740445e69 Sep 08 14:39:11.261496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-VolumesNegativeTest-1043166854-project Sep 08 14:39:11.261496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: Sep 08 14:39:11.261496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Volume 5801a83f-0f7d-4357-b436-6cd8e4ea7a35 could not be found. Sep 08 14:39:11.261496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': '5801a83f-0f7d-4357-b436-6cd8e4ea7a35'} Sep 08 14:39:11.261496 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 5801a83f-0f7d-4357-b436-6cd8e4ea7a35 could not be found. (HTTP 404) (Request-ID: req-e99e03b5-c11b-44d0-8010-f9796d8f141d) Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:11.262449 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 08 14:39:11.263714 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 5801a83f-0f7d-4357-b436-6cd8e4ea7a35 could not be found. Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 08 14:39:11.264824 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:39:11.265936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 5801a83f-0f7d-4357-b436-6cd8e4ea7a35 could not be found. Sep 08 14:39:11.265936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.265936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:11.265936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.265936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:11.265936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:39:11.265936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:39:11.265936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:39:11.265936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:39:11.265936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 5801a83f-0f7d-4357-b436-6cd8e4ea7a35 could not be found. Sep 08 14:39:11.265936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.265936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-422644ef-3e51-452c-b5c6-1c09f09104cc tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] Returning 404 to user: Volume 5801a83f-0f7d-4357-b436-6cd8e4ea7a35 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:39:11.265936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-422644ef-3e51-452c-b5c6-1c09f09104cc tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] 10.222.0.25 "DELETE /compute/v2.1/os-volumes/5801a83f-0f7d-4357-b436-6cd8e4ea7a35" status: 404 len: 109 microversion: 2.1 time: 0.089211 Sep 08 14:39:11.265936 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1584/2885] 10.222.0.25 () {58 vars in 1248 bytes} [Thu Sep 8 14:39:11 2022] DELETE /compute/v2.1/os-volumes/5801a83f-0f7d-4357-b436-6cd8e4ea7a35 => generated 109 bytes in 90 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:39:11.271166 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6d899c4d-cabd-4837-8390-464e17db8be9 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] Calling method '.version_select at 0x7fce021d25e0>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:39:11.299786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.wsgi [None req-6d899c4d-cabd-4837-8390-464e17db8be9 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] HTTP exception thrown: Volume 550dbf76-3b5d-4d27-a948-53e9aa486fc9 could not be found. Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6d899c4d-cabd-4837-8390-464e17db8be9 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] Request method failure captured: Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: request: GET /compute/v2.1/os-volumes/550dbf76-3b5d-4d27-a948-53e9aa486fc9 HTTP/1.1 Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept: application/json Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Accept-Encoding: identity Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Connection: close Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Length: 0 Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Content-Type: application/json Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Host: 10.222.0.25 Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: Openstack-System-Scope: None Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: User-Agent: python-urllib3/1.26.12 Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Auth-Token: gAAAAABjGf6OgdhQ_WRbJnJZ6wWxULuFHUcuMdscj7Jm7BR7IW1gM0fhHw62oo6kdOnfhyRJwbRIO6TuF8MTZK3l2fSB4M3uY5nP-syNijY8ekMcuns83Lq0TVrhvWPSiM2SV9BRpVD0Wz_fvaTH6kurL0dQe6KWWKBN7UuDVWVKVX-RfD4yZYY Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Id: None Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Domain-Name: None Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Identity-Status: Confirmed Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Is-Admin-Project: True Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Id: default Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Domain-Name: Default Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Id: 2c05b7c0d85a4a04a64aa3f9b25aa9bf Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Project-Name: tempest-VolumesNegativeTest-1043166854 Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Role: reader,member Sep 08 14:39:11.301715 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Roles: reader,member Sep 08 14:39:11.303147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/2c05b7c0d85a4a04a64aa3f9b25aa9bf"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:39:11.303147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant: tempest-VolumesNegativeTest-1043166854 Sep 08 14:39:11.303147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Id: 2c05b7c0d85a4a04a64aa3f9b25aa9bf Sep 08 14:39:11.303147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-Tenant-Name: tempest-VolumesNegativeTest-1043166854 Sep 08 14:39:11.303147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User: tempest-VolumesNegativeTest-1043166854-project Sep 08 14:39:11.303147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Id: default Sep 08 14:39:11.303147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Domain-Name: Default Sep 08 14:39:11.303147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Id: 437982b7ed6d47dd90e0232740445e69 Sep 08 14:39:11.303147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: X-User-Name: tempest-VolumesNegativeTest-1043166854-project Sep 08 14:39:11.303147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: method: Sep 08 14:39:11.303147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: exception: Volume 550dbf76-3b5d-4d27-a948-53e9aa486fc9 could not be found. Sep 08 14:39:11.303147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: action_args: {'id': '550dbf76-3b5d-4d27-a948-53e9aa486fc9'} Sep 08 14:39:11.303147 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 550dbf76-3b5d-4d27-a948-53e9aa486fc9 could not be found. (HTTP 404) (Request-ID: req-3867a87d-03a5-4732-8295-6b2f337e9ec0) Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:11.304007 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 08 14:39:11.305242 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 550dbf76-3b5d-4d27-a948-53e9aa486fc9 could not be found. Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 08 14:39:11.306848 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:39:11.308093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 550dbf76-3b5d-4d27-a948-53e9aa486fc9 could not be found. Sep 08 14:39:11.308093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.308093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:39:11.308093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.308093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:39:11.308093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:39:11.308093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:39:11.308093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:39:11.308093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:39:11.308093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 550dbf76-3b5d-4d27-a948-53e9aa486fc9 could not be found. Sep 08 14:39:11.308093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR nova.api.openstack.wsgi Sep 08 14:39:11.308093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-6d899c4d-cabd-4837-8390-464e17db8be9 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] Returning 404 to user: Volume 550dbf76-3b5d-4d27-a948-53e9aa486fc9 could not be found. {{(pid=127602) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:39:11.308093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-6d899c4d-cabd-4837-8390-464e17db8be9 tempest-VolumesNegativeTest-1043166854 tempest-VolumesNegativeTest-1043166854-project] 10.222.0.25 "GET /compute/v2.1/os-volumes/550dbf76-3b5d-4d27-a948-53e9aa486fc9" status: 404 len: 109 microversion: 2.1 time: 0.035873 Sep 08 14:39:11.308093 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1302/2886] 10.222.0.25 () {58 vars in 1245 bytes} [Thu Sep 8 14:39:11 2022] GET /compute/v2.1/os-volumes/550dbf76-3b5d-4d27-a948-53e9aa486fc9 => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 08 14:39:53.973510 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=2 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:53.974380 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=2 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:53.979020 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1484868231", "imageRef": "20b30425-92e5-4d78-8502-45557526ea49", "flavorRef": "452", "networks": [{"uuid": "ab1e4ea1-479d-4c14-89ff-e29a9d1155cc"}]}} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:39:54.032329 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.network.neutron [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] validate_networks() for [('ab1e4ea1-479d-4c14-89ff-e29a9d1155cc', None, None, None, None, None)] {{(pid=127601) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 14:39:54.439210 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Flavor limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 14:39:54.439675 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Image limits 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 14:39:54.440157 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Flavor pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 14:39:54.441101 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Image pref 0:0:0 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 14:39:54.441519 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.virt.hardware [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127601) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 14:39:54.450474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:39:54.450954 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:39:54.451394 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:39:54.452127 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:39:54.452583 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:39:54.453010 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:39:54.472054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Getting quotas for project 307bc84fa6ea41cda154754458eff813. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 14:39:54.476055 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.quota [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Getting quotas for user 08c895693aa54683951e91d4271c6191 and project 307bc84fa6ea41cda154754458eff813. Resources: {'cores', 'instances', 'ram'} {{(pid=127601) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 14:39:54.487288 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Going to run 1 instances... {{(pid=127601) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 14:39:54.492798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='20b30425-92e5-4d78-8502-45557526ea49',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127601) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 14:39:54.613913 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR oslo.messaging._drivers.impl_rabbit [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [fff75be0-8b12-4070-bb69-ea0ad3fec374] AMQP server on 10.222.0.25:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 08 14:39:55.621044 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [fff75be0-8b12-4070-bb69-ea0ad3fec374] Reconnected to AMQP server on 10.222.0.25:5672 via [amqp] client with port 44790. Sep 08 14:39:55.625359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4cac16ba-d0f9-4ec8-b66d-36f3ddda5b67 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.653846 Sep 08 14:39:55.626396 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1585/2887] 10.222.0.25 () {60 vars in 1147 bytes} [Thu Sep 8 14:39:53 2022] POST /compute/v2.1/servers => generated 380 bytes in 1655 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 08 14:39:55.635279 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=2 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:55.636586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=2 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:39:55.641330 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-80a4db11-5ca9-4cfe-acf9-e0470ccb3a66 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:39:55.641936 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-80a4db11-5ca9-4cfe-acf9-e0470ccb3a66 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:39:55.697676 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-80a4db11-5ca9-4cfe-acf9-e0470ccb3a66 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:39:55.704590 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-80a4db11-5ca9-4cfe-acf9-e0470ccb3a66 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e" status: 200 len: 1214 microversion: 2.1 time: 0.070667 Sep 08 14:39:55.705157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1303/2888] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:39:55 2022] GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e => generated 1214 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:39:56.729006 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-ab6ecb90-f05d-4f89-a371-713037322c7e tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:39:56.729655 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-ab6ecb90-f05d-4f89-a371-713037322c7e tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:39:56.788948 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-ab6ecb90-f05d-4f89-a371-713037322c7e tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:39:56.796877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-ab6ecb90-f05d-4f89-a371-713037322c7e tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e" status: 200 len: 1214 microversion: 2.1 time: 0.072227 Sep 08 14:39:56.797445 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1586/2889] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:39:56 2022] GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e => generated 1214 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:39:57.820958 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4542cac6-8673-4655-85fd-7abbc0dfe567 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:39:57.821582 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4542cac6-8673-4655-85fd-7abbc0dfe567 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:39:57.828756 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4542cac6-8673-4655-85fd-7abbc0dfe567 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:39:57.829309 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4542cac6-8673-4655-85fd-7abbc0dfe567 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:39:57.829744 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4542cac6-8673-4655-85fd-7abbc0dfe567 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:39:57.830493 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4542cac6-8673-4655-85fd-7abbc0dfe567 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:39:57.830949 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4542cac6-8673-4655-85fd-7abbc0dfe567 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:39:57.831372 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4542cac6-8673-4655-85fd-7abbc0dfe567 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:39:57.897812 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4542cac6-8673-4655-85fd-7abbc0dfe567 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:39:57.902926 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4542cac6-8673-4655-85fd-7abbc0dfe567 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e" status: 200 len: 1266 microversion: 2.1 time: 0.086536 Sep 08 14:39:57.903496 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1304/2890] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:39:57 2022] GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e => generated 1266 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:39:58.926558 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-987c2fde-f32e-4115-baa6-206df9e1aeeb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:39:58.927238 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-987c2fde-f32e-4115-baa6-206df9e1aeeb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:39:58.934416 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-987c2fde-f32e-4115-baa6-206df9e1aeeb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:39:58.934905 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-987c2fde-f32e-4115-baa6-206df9e1aeeb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:39:58.935345 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-987c2fde-f32e-4115-baa6-206df9e1aeeb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:39:58.936046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-987c2fde-f32e-4115-baa6-206df9e1aeeb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:39:58.936502 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-987c2fde-f32e-4115-baa6-206df9e1aeeb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:39:58.936939 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-987c2fde-f32e-4115-baa6-206df9e1aeeb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:39:59.000330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-987c2fde-f32e-4115-baa6-206df9e1aeeb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:39:59.008452 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-987c2fde-f32e-4115-baa6-206df9e1aeeb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e" status: 200 len: 1272 microversion: 2.1 time: 0.086105 Sep 08 14:39:59.009023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1587/2891] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:39:58 2022] GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e => generated 1272 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:00.032482 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-2567f89d-85c7-4575-a08d-ae3ad13601ae tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:00.032998 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-2567f89d-85c7-4575-a08d-ae3ad13601ae tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:00.040197 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2567f89d-85c7-4575-a08d-ae3ad13601ae tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:00.040682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2567f89d-85c7-4575-a08d-ae3ad13601ae tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:00.041116 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2567f89d-85c7-4575-a08d-ae3ad13601ae tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:00.041810 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2567f89d-85c7-4575-a08d-ae3ad13601ae tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:00.042294 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2567f89d-85c7-4575-a08d-ae3ad13601ae tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:00.042724 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-2567f89d-85c7-4575-a08d-ae3ad13601ae tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:00.074803 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-faca3fb8-0371-441e-bc98-7bc6212ab20f req-6468aff2-9c7b-42cf-9083-bb12ab2722c1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bfa7389c-c59b-4e80-85ea-109a03f5702e", "tag": "866113ad-97d7-4d36-9b4e-87e400c15ea9"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:40:00.090359 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-faca3fb8-0371-441e-bc98-7bc6212ab20f req-6468aff2-9c7b-42cf-9083-bb12ab2722c1 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:00.091124 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-faca3fb8-0371-441e-bc98-7bc6212ab20f req-6468aff2-9c7b-42cf-9083-bb12ab2722c1 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:00.091666 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-faca3fb8-0371-441e-bc98-7bc6212ab20f req-6468aff2-9c7b-42cf-9083-bb12ab2722c1 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:00.103501 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-faca3fb8-0371-441e-bc98-7bc6212ab20f req-6468aff2-9c7b-42cf-9083-bb12ab2722c1 service nova] Creating event network-changed:866113ad-97d7-4d36-9b4e-87e400c15ea9 for instance bfa7389c-c59b-4e80-85ea-109a03f5702e on hv-cinder-85646 Sep 08 14:40:00.106818 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR oslo.messaging._drivers.impl_rabbit [req-faca3fb8-0371-441e-bc98-7bc6212ab20f req-6468aff2-9c7b-42cf-9083-bb12ab2722c1 service nova] [4e313e95-054b-47a1-bc55-46b7309a8005] AMQP server on 10.222.0.25:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 08 14:40:00.175710 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-2567f89d-85c7-4575-a08d-ae3ad13601ae tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:00.183360 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-2567f89d-85c7-4575-a08d-ae3ad13601ae tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e" status: 200 len: 1314 microversion: 2.1 time: 0.155152 Sep 08 14:40:00.183926 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1305/2892] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:00 2022] GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e => generated 1314 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:01.114192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO oslo.messaging._drivers.impl_rabbit [req-faca3fb8-0371-441e-bc98-7bc6212ab20f req-6468aff2-9c7b-42cf-9083-bb12ab2722c1 service nova] [4e313e95-054b-47a1-bc55-46b7309a8005] Reconnected to AMQP server on 10.222.0.25:5672 via [amqp] client with port 44818. Sep 08 14:40:01.117915 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-faca3fb8-0371-441e-bc98-7bc6212ab20f req-6468aff2-9c7b-42cf-9083-bb12ab2722c1 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 1.055715 Sep 08 14:40:01.118554 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1588/2893] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:40:00 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 1058 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 08 14:40:01.206365 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-7bef8687-6c7b-430c-8eb6-6833e033cd09 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:01.206968 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-7bef8687-6c7b-430c-8eb6-6833e033cd09 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:01.213881 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7bef8687-6c7b-430c-8eb6-6833e033cd09 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:01.214392 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7bef8687-6c7b-430c-8eb6-6833e033cd09 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:01.214832 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7bef8687-6c7b-430c-8eb6-6833e033cd09 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:01.215512 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7bef8687-6c7b-430c-8eb6-6833e033cd09 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:01.215961 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7bef8687-6c7b-430c-8eb6-6833e033cd09 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:01.216378 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-7bef8687-6c7b-430c-8eb6-6833e033cd09 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:01.363343 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-7bef8687-6c7b-430c-8eb6-6833e033cd09 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:01.371054 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-7bef8687-6c7b-430c-8eb6-6833e033cd09 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e" status: 200 len: 1314 microversion: 2.1 time: 0.168521 Sep 08 14:40:01.371620 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1306/2894] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:01 2022] GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e => generated 1314 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:02.394652 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-0bd0e039-1e07-45a8-8a51-0dc5098ec9fa tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:02.395364 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-0bd0e039-1e07-45a8-8a51-0dc5098ec9fa tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:02.402286 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0bd0e039-1e07-45a8-8a51-0dc5098ec9fa tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:02.402780 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0bd0e039-1e07-45a8-8a51-0dc5098ec9fa tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:02.403225 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0bd0e039-1e07-45a8-8a51-0dc5098ec9fa tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:02.403929 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0bd0e039-1e07-45a8-8a51-0dc5098ec9fa tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:02.404645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0bd0e039-1e07-45a8-8a51-0dc5098ec9fa tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:02.405081 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-0bd0e039-1e07-45a8-8a51-0dc5098ec9fa tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:02.536684 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-0bd0e039-1e07-45a8-8a51-0dc5098ec9fa tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:02.544658 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-0bd0e039-1e07-45a8-8a51-0dc5098ec9fa tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e" status: 200 len: 1314 microversion: 2.1 time: 0.154721 Sep 08 14:40:02.545232 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1589/2895] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:02 2022] GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e => generated 1314 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:03.567158 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-ed78c775-a04b-439c-9989-626e3d0d0427 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:03.567771 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-ed78c775-a04b-439c-9989-626e3d0d0427 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:03.574517 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed78c775-a04b-439c-9989-626e3d0d0427 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:03.574999 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed78c775-a04b-439c-9989-626e3d0d0427 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:03.575435 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed78c775-a04b-439c-9989-626e3d0d0427 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:03.576118 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed78c775-a04b-439c-9989-626e3d0d0427 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:03.576570 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed78c775-a04b-439c-9989-626e3d0d0427 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:03.576996 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-ed78c775-a04b-439c-9989-626e3d0d0427 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:03.713684 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-ed78c775-a04b-439c-9989-626e3d0d0427 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:03.721554 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-ed78c775-a04b-439c-9989-626e3d0d0427 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e" status: 200 len: 1314 microversion: 2.1 time: 0.158263 Sep 08 14:40:03.722239 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1307/2896] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:03 2022] GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e => generated 1314 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:04.745887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-175b3169-d6c7-42d9-a814-6c4fd0f6705a tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:04.746526 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-175b3169-d6c7-42d9-a814-6c4fd0f6705a tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:04.748920 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=2 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:40:04.753734 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-175b3169-d6c7-42d9-a814-6c4fd0f6705a tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:04.754264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-175b3169-d6c7-42d9-a814-6c4fd0f6705a tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:04.754712 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-175b3169-d6c7-42d9-a814-6c4fd0f6705a tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:04.755410 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-175b3169-d6c7-42d9-a814-6c4fd0f6705a tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:04.755910 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-175b3169-d6c7-42d9-a814-6c4fd0f6705a tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:04.756346 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-175b3169-d6c7-42d9-a814-6c4fd0f6705a tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:04.882975 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-175b3169-d6c7-42d9-a814-6c4fd0f6705a tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:04.890596 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-175b3169-d6c7-42d9-a814-6c4fd0f6705a tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e" status: 200 len: 1314 microversion: 2.1 time: 0.148523 Sep 08 14:40:04.891099 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1590/2897] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:04 2022] GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e => generated 1314 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:05.213680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-5486b9d8-2f30-4731-bfba-296f1c26658f req-874c7fae-ab5d-4ac7-916e-47d6996b9e2f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bfa7389c-c59b-4e80-85ea-109a03f5702e", "name": "network-vif-plugged", "status": "completed", "tag": "866113ad-97d7-4d36-9b4e-87e400c15ea9"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:40:05.223560 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-5486b9d8-2f30-4731-bfba-296f1c26658f req-874c7fae-ab5d-4ac7-916e-47d6996b9e2f service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:05.224034 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-5486b9d8-2f30-4731-bfba-296f1c26658f req-874c7fae-ab5d-4ac7-916e-47d6996b9e2f service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:05.224474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-5486b9d8-2f30-4731-bfba-296f1c26658f req-874c7fae-ab5d-4ac7-916e-47d6996b9e2f service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:05.235773 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-5486b9d8-2f30-4731-bfba-296f1c26658f req-874c7fae-ab5d-4ac7-916e-47d6996b9e2f service nova] Creating event network-vif-plugged:866113ad-97d7-4d36-9b4e-87e400c15ea9 for instance bfa7389c-c59b-4e80-85ea-109a03f5702e on hv-cinder-85646 Sep 08 14:40:05.239127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR oslo.messaging._drivers.impl_rabbit [req-5486b9d8-2f30-4731-bfba-296f1c26658f req-874c7fae-ab5d-4ac7-916e-47d6996b9e2f service nova] [5c3268a0-88fa-4840-ae7d-d31840c7ba9f] AMQP server on 10.222.0.25:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 08 14:40:05.701897 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=2 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:40:05.913963 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-2e4ae410-4dab-4d51-8033-a45336ead7a5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:05.914555 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-2e4ae410-4dab-4d51-8033-a45336ead7a5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:05.920932 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e4ae410-4dab-4d51-8033-a45336ead7a5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:05.921417 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e4ae410-4dab-4d51-8033-a45336ead7a5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:05.921856 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e4ae410-4dab-4d51-8033-a45336ead7a5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:05.922579 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e4ae410-4dab-4d51-8033-a45336ead7a5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:05.923037 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e4ae410-4dab-4d51-8033-a45336ead7a5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:05.923462 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-2e4ae410-4dab-4d51-8033-a45336ead7a5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:06.054222 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-2e4ae410-4dab-4d51-8033-a45336ead7a5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:06.061840 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-2e4ae410-4dab-4d51-8033-a45336ead7a5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e" status: 200 len: 1314 microversion: 2.1 time: 0.152389 Sep 08 14:40:06.062437 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1591/2898] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:05 2022] GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e => generated 1314 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:40:06.245064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [req-5486b9d8-2f30-4731-bfba-296f1c26658f req-874c7fae-ab5d-4ac7-916e-47d6996b9e2f service nova] [5c3268a0-88fa-4840-ae7d-d31840c7ba9f] Reconnected to AMQP server on 10.222.0.25:5672 via [amqp] client with port 44832. Sep 08 14:40:06.248333 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-5486b9d8-2f30-4731-bfba-296f1c26658f req-874c7fae-ab5d-4ac7-916e-47d6996b9e2f service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 1.039381 Sep 08 14:40:06.248935 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1308/2899] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:40:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 1041 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:40:07.085478 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f41c44d8-0a14-4c49-a8e9-c1be6393ec3f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:07.086087 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f41c44d8-0a14-4c49-a8e9-c1be6393ec3f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:07.087982 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=8 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:40:07.093379 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f41c44d8-0a14-4c49-a8e9-c1be6393ec3f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:07.093862 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f41c44d8-0a14-4c49-a8e9-c1be6393ec3f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:07.094334 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f41c44d8-0a14-4c49-a8e9-c1be6393ec3f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:07.098372 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f41c44d8-0a14-4c49-a8e9-c1be6393ec3f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:07.098372 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f41c44d8-0a14-4c49-a8e9-c1be6393ec3f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:07.098372 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f41c44d8-0a14-4c49-a8e9-c1be6393ec3f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:07.226845 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-f41c44d8-0a14-4c49-a8e9-c1be6393ec3f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:07.234492 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f41c44d8-0a14-4c49-a8e9-c1be6393ec3f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e" status: 200 len: 1314 microversion: 2.1 time: 0.153516 Sep 08 14:40:07.235054 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1592/2900] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:07 2022] GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e => generated 1314 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:08.259217 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-296666bb-f029-414f-b116-e549f525e4db tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:08.259917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-296666bb-f029-414f-b116-e549f525e4db tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:08.266157 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-296666bb-f029-414f-b116-e549f525e4db tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:08.266633 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-296666bb-f029-414f-b116-e549f525e4db tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:08.267060 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-296666bb-f029-414f-b116-e549f525e4db tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:08.267736 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-296666bb-f029-414f-b116-e549f525e4db tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:08.268182 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-296666bb-f029-414f-b116-e549f525e4db tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:08.268606 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-296666bb-f029-414f-b116-e549f525e4db tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:08.401301 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-296666bb-f029-414f-b116-e549f525e4db tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:08.409680 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-296666bb-f029-414f-b116-e549f525e4db tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e" status: 200 len: 1314 microversion: 2.1 time: 0.155782 Sep 08 14:40:08.410279 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1309/2901] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:08 2022] GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e => generated 1314 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:09.433797 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-54316b8b-fa7c-4735-9ad3-533b2e9d2a70 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:09.434385 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-54316b8b-fa7c-4735-9ad3-533b2e9d2a70 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:09.440709 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-54316b8b-fa7c-4735-9ad3-533b2e9d2a70 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:09.441196 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-54316b8b-fa7c-4735-9ad3-533b2e9d2a70 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:09.441632 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-54316b8b-fa7c-4735-9ad3-533b2e9d2a70 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:09.442425 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-54316b8b-fa7c-4735-9ad3-533b2e9d2a70 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:09.442887 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-54316b8b-fa7c-4735-9ad3-533b2e9d2a70 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:09.443988 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-54316b8b-fa7c-4735-9ad3-533b2e9d2a70 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:09.572073 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-54316b8b-fa7c-4735-9ad3-533b2e9d2a70 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:09.579747 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-54316b8b-fa7c-4735-9ad3-533b2e9d2a70 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e" status: 200 len: 1498 microversion: 2.1 time: 0.149969 Sep 08 14:40:09.580310 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1593/2902] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:09 2022] GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e => generated 1498 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:10.515450 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e9c74626-bb6d-4bdb-9217-933888b648f5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:10.516249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e9c74626-bb6d-4bdb-9217-933888b648f5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:10.522573 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e9c74626-bb6d-4bdb-9217-933888b648f5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:10.523056 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e9c74626-bb6d-4bdb-9217-933888b648f5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:10.523488 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e9c74626-bb6d-4bdb-9217-933888b648f5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:10.524227 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e9c74626-bb6d-4bdb-9217-933888b648f5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:10.524682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e9c74626-bb6d-4bdb-9217-933888b648f5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:10.525105 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e9c74626-bb6d-4bdb-9217-933888b648f5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:10.551404 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e9c74626-bb6d-4bdb-9217-933888b648f5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:40:10.905922 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e9c74626-bb6d-4bdb-9217-933888b648f5 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "DELETE /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e" status: 204 len: 0 microversion: 2.1 time: 0.396419 Sep 08 14:40:10.906708 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1310/2903] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:40:10 2022] DELETE /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e => generated 0 bytes in 398 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:40:10.918405 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-657efe9c-08cd-40a7-978a-79680f9a5952 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:10.919192 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-657efe9c-08cd-40a7-978a-79680f9a5952 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:10.928668 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-657efe9c-08cd-40a7-978a-79680f9a5952 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:10.932521 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-657efe9c-08cd-40a7-978a-79680f9a5952 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:10.932521 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-657efe9c-08cd-40a7-978a-79680f9a5952 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:10.932521 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-657efe9c-08cd-40a7-978a-79680f9a5952 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:10.932521 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-657efe9c-08cd-40a7-978a-79680f9a5952 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:10.932836 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-657efe9c-08cd-40a7-978a-79680f9a5952 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:11.076510 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-657efe9c-08cd-40a7-978a-79680f9a5952 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:11.085476 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-657efe9c-08cd-40a7-978a-79680f9a5952 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e" status: 200 len: 1504 microversion: 2.1 time: 0.173398 Sep 08 14:40:11.086073 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1594/2904] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:10 2022] GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e => generated 1504 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:12.111120 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c72285d0-01fe-4b5b-835d-3b9082e6378d tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:12.111833 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c72285d0-01fe-4b5b-835d-3b9082e6378d tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:12.118682 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c72285d0-01fe-4b5b-835d-3b9082e6378d tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:12.119165 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c72285d0-01fe-4b5b-835d-3b9082e6378d tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:12.119595 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c72285d0-01fe-4b5b-835d-3b9082e6378d tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:12.120322 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c72285d0-01fe-4b5b-835d-3b9082e6378d tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:12.120774 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c72285d0-01fe-4b5b-835d-3b9082e6378d tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:12.121198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c72285d0-01fe-4b5b-835d-3b9082e6378d tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:12.264930 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c72285d0-01fe-4b5b-835d-3b9082e6378d tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:12.273599 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c72285d0-01fe-4b5b-835d-3b9082e6378d tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e" status: 200 len: 1504 microversion: 2.1 time: 0.167384 Sep 08 14:40:12.279714 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1311/2905] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:12 2022] GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e => generated 1504 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:13.117403 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-8b25fe25-0add-4bd1-99ae-5a3e9625e914 req-959fcf83-b4ae-439b-bc10-1ed051f726cd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bfa7389c-c59b-4e80-85ea-109a03f5702e", "name": "network-vif-deleted", "tag": "866113ad-97d7-4d36-9b4e-87e400c15ea9"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:40:13.128256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-8b25fe25-0add-4bd1-99ae-5a3e9625e914 req-959fcf83-b4ae-439b-bc10-1ed051f726cd service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:13.128256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-8b25fe25-0add-4bd1-99ae-5a3e9625e914 req-959fcf83-b4ae-439b-bc10-1ed051f726cd service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:13.128256 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-8b25fe25-0add-4bd1-99ae-5a3e9625e914 req-959fcf83-b4ae-439b-bc10-1ed051f726cd service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:13.141970 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-8b25fe25-0add-4bd1-99ae-5a3e9625e914 req-959fcf83-b4ae-439b-bc10-1ed051f726cd service nova] Creating event network-vif-deleted:866113ad-97d7-4d36-9b4e-87e400c15ea9 for instance bfa7389c-c59b-4e80-85ea-109a03f5702e on hv-cinder-85646 Sep 08 14:40:13.147198 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-8b25fe25-0add-4bd1-99ae-5a3e9625e914 req-959fcf83-b4ae-439b-bc10-1ed051f726cd service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034832 Sep 08 14:40:13.147798 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1595/2906] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:40:13 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:40:13.298363 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-41d0d3f5-64d6-46ed-9785-1685deb69c20 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:13.298963 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-41d0d3f5-64d6-46ed-9785-1685deb69c20 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:13.305951 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-41d0d3f5-64d6-46ed-9785-1685deb69c20 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:13.308249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-41d0d3f5-64d6-46ed-9785-1685deb69c20 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:13.308249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-41d0d3f5-64d6-46ed-9785-1685deb69c20 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:13.308249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-41d0d3f5-64d6-46ed-9785-1685deb69c20 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:13.308249 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-41d0d3f5-64d6-46ed-9785-1685deb69c20 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:13.308724 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-41d0d3f5-64d6-46ed-9785-1685deb69c20 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:13.372223 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-41d0d3f5-64d6-46ed-9785-1685deb69c20 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:13.381264 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-41d0d3f5-64d6-46ed-9785-1685deb69c20 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e" status: 200 len: 1299 microversion: 2.1 time: 0.086681 Sep 08 14:40:13.383094 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1312/2907] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:13 2022] GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e => generated 1299 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:14.404674 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-66783659-d3c0-4c87-a998-d676d9f54147 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:14.405374 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-66783659-d3c0-4c87-a998-d676d9f54147 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: bfa7389c-c59b-4e80-85ea-109a03f5702e] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:14.411517 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66783659-d3c0-4c87-a998-d676d9f54147 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:14.412520 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66783659-d3c0-4c87-a998-d676d9f54147 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:14.412964 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66783659-d3c0-4c87-a998-d676d9f54147 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:14.413645 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66783659-d3c0-4c87-a998-d676d9f54147 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:14.414097 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66783659-d3c0-4c87-a998-d676d9f54147 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:14.414543 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-66783659-d3c0-4c87-a998-d676d9f54147 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:14.428472 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-66783659-d3c0-4c87-a998-d676d9f54147 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] HTTP exception thrown: Instance bfa7389c-c59b-4e80-85ea-109a03f5702e could not be found. Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-66783659-d3c0-4c87-a998-d676d9f54147 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Request method failure captured: Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e HTTP/1.1 Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGf63izntBoSNvhJPI_fmsgIWS0vzcbmBFO5a1EorMoSh53FWRxht9zuF6-AtdBe-r8DZh1-lfLeu8LN7PbpNf3abTp7sIzfM_BQJeIFS2NQH6wyHcQCuAQoIo4bXYyCv6wHU62sQp4bF6c27CXZbLk0X0dlM3CcZq-C78eNqBkqeHdM Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 307bc84fa6ea41cda154754458eff813 Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-VolumesActionsTest-369921482 Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:40:14.430160 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:40:14.433665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_307bc84fa6ea41cda154754458eff813"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/307bc84fa6ea41cda154754458eff813"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/307bc84fa6ea41cda154754458eff813"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/307bc84fa6ea41cda154754458eff813"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:40:14.433665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-VolumesActionsTest-369921482 Sep 08 14:40:14.433665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 307bc84fa6ea41cda154754458eff813 Sep 08 14:40:14.433665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-VolumesActionsTest-369921482 Sep 08 14:40:14.433665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-VolumesActionsTest-369921482-project Sep 08 14:40:14.433665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:40:14.433665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:40:14.433665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 08c895693aa54683951e91d4271c6191 Sep 08 14:40:14.433665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-VolumesActionsTest-369921482-project Sep 08 14:40:14.433665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:40:14.433665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance bfa7389c-c59b-4e80-85ea-109a03f5702e could not be found. Sep 08 14:40:14.433665 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': 'bfa7389c-c59b-4e80-85ea-109a03f5702e'} Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bfa7389c-c59b-4e80-85ea-109a03f5702e could not be found. Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:40:14.434598 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:40:14.435980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:40:14.435980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:40:14.435980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:40:14.435980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:40:14.435980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:40:14.435980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:40:14.435980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:40:14.435980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bfa7389c-c59b-4e80-85ea-109a03f5702e could not be found. Sep 08 14:40:14.435980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:40:14.435980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:40:14.435980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:40:14.435980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:40:14.435980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:40:14.435980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:40:14.435980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:40:14.435980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:40:14.435980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bfa7389c-c59b-4e80-85ea-109a03f5702e could not be found. Sep 08 14:40:14.435980 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:40:14.437226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-66783659-d3c0-4c87-a998-d676d9f54147 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Returning 404 to user: Instance bfa7389c-c59b-4e80-85ea-109a03f5702e could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:40:14.437226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-66783659-d3c0-4c87-a998-d676d9f54147 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e" status: 404 len: 111 microversion: 2.1 time: 0.031859 Sep 08 14:40:14.437226 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1596/2908] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:14 2022] GET /compute/v2.1/servers/bfa7389c-c59b-4e80-85ea-109a03f5702e => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 08 14:40:14.481473 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-475275914", "imageRef": "20b30425-92e5-4d78-8502-45557526ea49", "flavorRef": "452", "networks": [{"uuid": "ab1e4ea1-479d-4c14-89ff-e29a9d1155cc"}]}} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:40:14.539500 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.network.neutron [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] validate_networks() for [('ab1e4ea1-479d-4c14-89ff-e29a9d1155cc', None, None, None, None, None)] {{(pid=127602) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 08 14:40:14.966825 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Flavor limits 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 08 14:40:14.967243 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Image limits 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 08 14:40:14.967717 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Flavor pref 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 08 14:40:14.968123 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Image pref 0:0:0 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 08 14:40:14.968526 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.virt.hardware [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=127602) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 08 14:40:14.977754 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:14.978327 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:14.978775 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:14.979474 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:14.980021 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:14.980440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:14.996763 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Getting quotas for project 307bc84fa6ea41cda154754458eff813. Resources: {'cores', 'instances', 'ram'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 08 14:40:15.000684 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.quota [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Getting quotas for user 08c895693aa54683951e91d4271c6191 and project 307bc84fa6ea41cda154754458eff813. Resources: {'cores', 'instances', 'ram'} {{(pid=127602) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 08 14:40:15.011632 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Going to run 1 instances... {{(pid=127602) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 08 14:40:15.013877 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='20b30425-92e5-4d78-8502-45557526ea49',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=127602) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 08 14:40:15.160647 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: ERROR oslo.messaging._drivers.impl_rabbit [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [4582b676-369a-4a0a-b2b9-b7eb1d1a1bbf] AMQP server on 10.222.0.25:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 08 14:40:16.168650 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO oslo.messaging._drivers.impl_rabbit [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [4582b676-369a-4a0a-b2b9-b7eb1d1a1bbf] Reconnected to AMQP server on 10.222.0.25:5672 via [amqp] client with port 44874. Sep 08 14:40:16.172989 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-88380225-def8-4a22-a9a5-38e1bd924dfe tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.695377 Sep 08 14:40:16.173806 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1313/2909] 10.222.0.25 () {60 vars in 1147 bytes} [Thu Sep 8 14:40:14 2022] POST /compute/v2.1/servers => generated 380 bytes in 1697 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 08 14:40:16.185274 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-4f9b1f5e-512b-4728-95d7-cc0c91271b14 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:16.185877 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-4f9b1f5e-512b-4728-95d7-cc0c91271b14 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:16.242080 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-4f9b1f5e-512b-4728-95d7-cc0c91271b14 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:16.249888 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-4f9b1f5e-512b-4728-95d7-cc0c91271b14 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 200 len: 1213 microversion: 2.1 time: 0.068507 Sep 08 14:40:16.250726 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1597/2910] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:16 2022] GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 1213 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:17.273780 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-cde0bb20-425d-411b-8419-61c1dffa37fb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:17.274425 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-cde0bb20-425d-411b-8419-61c1dffa37fb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:17.280783 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cde0bb20-425d-411b-8419-61c1dffa37fb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:17.281266 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cde0bb20-425d-411b-8419-61c1dffa37fb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:17.281702 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cde0bb20-425d-411b-8419-61c1dffa37fb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:17.282415 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cde0bb20-425d-411b-8419-61c1dffa37fb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:17.282866 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cde0bb20-425d-411b-8419-61c1dffa37fb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:17.283290 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-cde0bb20-425d-411b-8419-61c1dffa37fb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:17.351344 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-cde0bb20-425d-411b-8419-61c1dffa37fb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:17.359123 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-cde0bb20-425d-411b-8419-61c1dffa37fb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 200 len: 1217 microversion: 2.1 time: 0.090061 Sep 08 14:40:17.359694 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1314/2911] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:17 2022] GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 1217 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:18.383183 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-6c72399e-6526-4dcb-a6fa-2df0ac385673 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:18.383793 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-6c72399e-6526-4dcb-a6fa-2df0ac385673 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:18.391588 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c72399e-6526-4dcb-a6fa-2df0ac385673 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:18.400309 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c72399e-6526-4dcb-a6fa-2df0ac385673 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:18.400309 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c72399e-6526-4dcb-a6fa-2df0ac385673 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:18.400309 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c72399e-6526-4dcb-a6fa-2df0ac385673 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:18.400309 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c72399e-6526-4dcb-a6fa-2df0ac385673 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:18.400724 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-6c72399e-6526-4dcb-a6fa-2df0ac385673 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:18.463773 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-6c72399e-6526-4dcb-a6fa-2df0ac385673 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:18.471685 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-6c72399e-6526-4dcb-a6fa-2df0ac385673 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 200 len: 1265 microversion: 2.1 time: 0.092859 Sep 08 14:40:18.472355 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1598/2912] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:18 2022] GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 1265 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:19.495215 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-4f45c867-e1bc-4fe5-8313-8009192dae48 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:19.495783 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-4f45c867-e1bc-4fe5-8313-8009192dae48 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:19.502913 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f45c867-e1bc-4fe5-8313-8009192dae48 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:19.503394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f45c867-e1bc-4fe5-8313-8009192dae48 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:19.503823 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f45c867-e1bc-4fe5-8313-8009192dae48 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:19.504516 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f45c867-e1bc-4fe5-8313-8009192dae48 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:19.504972 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f45c867-e1bc-4fe5-8313-8009192dae48 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:19.505398 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-4f45c867-e1bc-4fe5-8313-8009192dae48 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:19.653305 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-4f45c867-e1bc-4fe5-8313-8009192dae48 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:19.661603 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-4f45c867-e1bc-4fe5-8313-8009192dae48 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 200 len: 1313 microversion: 2.1 time: 0.170321 Sep 08 14:40:19.662268 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1315/2913] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:19 2022] GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 1313 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:20.390816 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-9bebc21a-ef29-4bff-99d7-d98f278ed7bf req-1b00f9b6-ebe1-4550-9659-19520ba74195 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "29c8d7e2-687b-4169-95da-4446901ffd51", "tag": "85bc17ba-5320-42c0-a869-d4017b754772"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:40:20.400643 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-9bebc21a-ef29-4bff-99d7-d98f278ed7bf req-1b00f9b6-ebe1-4550-9659-19520ba74195 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:20.401123 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-9bebc21a-ef29-4bff-99d7-d98f278ed7bf req-1b00f9b6-ebe1-4550-9659-19520ba74195 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:20.401555 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-9bebc21a-ef29-4bff-99d7-d98f278ed7bf req-1b00f9b6-ebe1-4550-9659-19520ba74195 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:20.413610 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-9bebc21a-ef29-4bff-99d7-d98f278ed7bf req-1b00f9b6-ebe1-4550-9659-19520ba74195 service nova] Creating event network-changed:85bc17ba-5320-42c0-a869-d4017b754772 for instance 29c8d7e2-687b-4169-95da-4446901ffd51 on hv-cinder-85646 Sep 08 14:40:20.418584 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-9bebc21a-ef29-4bff-99d7-d98f278ed7bf req-1b00f9b6-ebe1-4550-9659-19520ba74195 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.032280 Sep 08 14:40:20.419186 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1599/2914] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:40:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:40:20.684770 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-c9f35197-71e3-445a-a56f-4ed70ade9064 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:20.685489 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-c9f35197-71e3-445a-a56f-4ed70ade9064 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:20.692178 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c9f35197-71e3-445a-a56f-4ed70ade9064 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:20.692760 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c9f35197-71e3-445a-a56f-4ed70ade9064 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:20.693198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c9f35197-71e3-445a-a56f-4ed70ade9064 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:20.693893 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c9f35197-71e3-445a-a56f-4ed70ade9064 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:20.694391 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c9f35197-71e3-445a-a56f-4ed70ade9064 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:20.694851 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-c9f35197-71e3-445a-a56f-4ed70ade9064 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:20.834356 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-c9f35197-71e3-445a-a56f-4ed70ade9064 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:20.842479 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-c9f35197-71e3-445a-a56f-4ed70ade9064 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 200 len: 1313 microversion: 2.1 time: 0.161521 Sep 08 14:40:20.843069 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1316/2915] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:20 2022] GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 1313 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:21.867599 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-7c13e137-f2e4-4ded-81d4-555edccd6eb0 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:21.868307 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-7c13e137-f2e4-4ded-81d4-555edccd6eb0 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:21.874693 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7c13e137-f2e4-4ded-81d4-555edccd6eb0 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:21.875182 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7c13e137-f2e4-4ded-81d4-555edccd6eb0 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:21.875647 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7c13e137-f2e4-4ded-81d4-555edccd6eb0 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:21.876336 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7c13e137-f2e4-4ded-81d4-555edccd6eb0 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:21.876795 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7c13e137-f2e4-4ded-81d4-555edccd6eb0 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:21.877221 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-7c13e137-f2e4-4ded-81d4-555edccd6eb0 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:22.008754 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-7c13e137-f2e4-4ded-81d4-555edccd6eb0 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:22.016531 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-7c13e137-f2e4-4ded-81d4-555edccd6eb0 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 200 len: 1313 microversion: 2.1 time: 0.154545 Sep 08 14:40:22.020772 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1600/2916] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:21 2022] GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 1313 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:23.039911 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-a7dba79d-aea2-4cba-8427-70107f3e5e15 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:23.040840 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-a7dba79d-aea2-4cba-8427-70107f3e5e15 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:23.047127 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7dba79d-aea2-4cba-8427-70107f3e5e15 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:23.047617 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7dba79d-aea2-4cba-8427-70107f3e5e15 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:23.048112 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7dba79d-aea2-4cba-8427-70107f3e5e15 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:23.048810 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7dba79d-aea2-4cba-8427-70107f3e5e15 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:23.049267 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7dba79d-aea2-4cba-8427-70107f3e5e15 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:23.049691 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-a7dba79d-aea2-4cba-8427-70107f3e5e15 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:23.186031 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-a7dba79d-aea2-4cba-8427-70107f3e5e15 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:23.194349 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-a7dba79d-aea2-4cba-8427-70107f3e5e15 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 200 len: 1313 microversion: 2.1 time: 0.158580 Sep 08 14:40:23.194920 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1317/2917] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:23 2022] GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 1313 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:24.216118 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-d79ffcfe-5c46-46a0-a05e-d119d1fd2afb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:24.216767 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-d79ffcfe-5c46-46a0-a05e-d119d1fd2afb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:24.219230 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_api:SELECT=48,nova_api:INSERT=3 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:40:24.224248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d79ffcfe-5c46-46a0-a05e-d119d1fd2afb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:24.224737 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d79ffcfe-5c46-46a0-a05e-d119d1fd2afb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:24.225214 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d79ffcfe-5c46-46a0-a05e-d119d1fd2afb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:24.225917 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d79ffcfe-5c46-46a0-a05e-d119d1fd2afb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:24.226479 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d79ffcfe-5c46-46a0-a05e-d119d1fd2afb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:24.226907 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-d79ffcfe-5c46-46a0-a05e-d119d1fd2afb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:24.229557 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell1:SELECT=65 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:40:24.362000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-d79ffcfe-5c46-46a0-a05e-d119d1fd2afb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:24.370348 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-d79ffcfe-5c46-46a0-a05e-d119d1fd2afb tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 200 len: 1313 microversion: 2.1 time: 0.158071 Sep 08 14:40:24.370914 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1601/2918] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:24 2022] GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 1313 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:25.392148 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-e7cbd283-198f-46c5-a4b6-eb0776f54b87 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:25.393198 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-e7cbd283-198f-46c5-a4b6-eb0776f54b87 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:25.395677 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=2 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:40:25.396331 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell0:SELECT=6 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:40:25.401695 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7cbd283-198f-46c5-a4b6-eb0776f54b87 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:25.402207 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7cbd283-198f-46c5-a4b6-eb0776f54b87 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:25.402750 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7cbd283-198f-46c5-a4b6-eb0776f54b87 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:25.403435 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7cbd283-198f-46c5-a4b6-eb0776f54b87 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:25.403871 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7cbd283-198f-46c5-a4b6-eb0776f54b87 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:25.410789 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-e7cbd283-198f-46c5-a4b6-eb0776f54b87 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:25.538172 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-e7cbd283-198f-46c5-a4b6-eb0776f54b87 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:25.545853 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-e7cbd283-198f-46c5-a4b6-eb0776f54b87 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 200 len: 1313 microversion: 2.1 time: 0.157620 Sep 08 14:40:25.546475 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1318/2919] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:25 2022] GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 1313 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:26.569096 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-1b51c804-f3e5-4b39-9404-d5c969ea998f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:26.569706 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-1b51c804-f3e5-4b39-9404-d5c969ea998f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:26.571998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG dbcounter [-] [127601] Writing DB stats nova_cell0:SELECT=2 {{(pid=127601) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:40:26.577938 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1b51c804-f3e5-4b39-9404-d5c969ea998f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:26.578482 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1b51c804-f3e5-4b39-9404-d5c969ea998f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:26.586673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1b51c804-f3e5-4b39-9404-d5c969ea998f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:26.586673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1b51c804-f3e5-4b39-9404-d5c969ea998f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:26.586673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1b51c804-f3e5-4b39-9404-d5c969ea998f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:26.586673 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-1b51c804-f3e5-4b39-9404-d5c969ea998f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:26.713434 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-1b51c804-f3e5-4b39-9404-d5c969ea998f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:26.721158 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-1b51c804-f3e5-4b39-9404-d5c969ea998f tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 200 len: 1313 microversion: 2.1 time: 0.156077 Sep 08 14:40:26.721730 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1602/2920] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:26 2022] GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 1313 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:27.348447 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [req-542509c2-2970-4674-86b0-429d3a33632d req-794201e9-c839-47ae-90c1-27cffda466f5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "29c8d7e2-687b-4169-95da-4446901ffd51", "name": "network-vif-plugged", "status": "completed", "tag": "85bc17ba-5320-42c0-a869-d4017b754772"}]} {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:40:27.359177 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-542509c2-2970-4674-86b0-429d3a33632d req-794201e9-c839-47ae-90c1-27cffda466f5 service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:27.362612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-542509c2-2970-4674-86b0-429d3a33632d req-794201e9-c839-47ae-90c1-27cffda466f5 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:27.362612 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [req-542509c2-2970-4674-86b0-429d3a33632d req-794201e9-c839-47ae-90c1-27cffda466f5 service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:27.363925 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:40:27.371647 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.compute.server_external_events [req-542509c2-2970-4674-86b0-429d3a33632d req-794201e9-c839-47ae-90c1-27cffda466f5 service nova] Creating event network-vif-plugged:85bc17ba-5320-42c0-a869-d4017b754772 for instance 29c8d7e2-687b-4169-95da-4446901ffd51 on hv-cinder-85646 Sep 08 14:40:27.376721 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [req-542509c2-2970-4674-86b0-429d3a33632d req-794201e9-c839-47ae-90c1-27cffda466f5 service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032300 Sep 08 14:40:27.377307 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1319/2921] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:40:27 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:40:27.744239 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-8868445c-9844-4418-b382-e36da44bddcd tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:27.744782 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-8868445c-9844-4418-b382-e36da44bddcd tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:27.751248 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8868445c-9844-4418-b382-e36da44bddcd tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:27.751813 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8868445c-9844-4418-b382-e36da44bddcd tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:27.752264 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8868445c-9844-4418-b382-e36da44bddcd tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:27.752969 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8868445c-9844-4418-b382-e36da44bddcd tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:27.753430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8868445c-9844-4418-b382-e36da44bddcd tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:27.753864 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-8868445c-9844-4418-b382-e36da44bddcd tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:27.921324 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-8868445c-9844-4418-b382-e36da44bddcd tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:27.928989 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-8868445c-9844-4418-b382-e36da44bddcd tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 200 len: 1313 microversion: 2.1 time: 0.188826 Sep 08 14:40:27.929564 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1603/2922] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:27 2022] GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 1313 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:28.953096 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1de5d3c1-5438-4640-915f-e299a4e4f6f6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:28.953714 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1de5d3c1-5438-4640-915f-e299a4e4f6f6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:28.955689 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG dbcounter [-] [127602] Writing DB stats nova_api:SELECT=53,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=127602) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:40:28.961430 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1de5d3c1-5438-4640-915f-e299a4e4f6f6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:28.961917 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1de5d3c1-5438-4640-915f-e299a4e4f6f6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:28.962382 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1de5d3c1-5438-4640-915f-e299a4e4f6f6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:28.963090 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1de5d3c1-5438-4640-915f-e299a4e4f6f6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:28.963539 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1de5d3c1-5438-4640-915f-e299a4e4f6f6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:28.963965 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1de5d3c1-5438-4640-915f-e299a4e4f6f6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:29.103244 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1de5d3c1-5438-4640-915f-e299a4e4f6f6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:29.110819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1de5d3c1-5438-4640-915f-e299a4e4f6f6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 200 len: 1313 microversion: 2.1 time: 0.162945 Sep 08 14:40:29.111386 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1320/2923] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:28 2022] GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 1313 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:30.133478 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-67b513e1-ed2c-4c13-bc34-941a3f72868e tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:30.134115 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-67b513e1-ed2c-4c13-bc34-941a3f72868e tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:30.140842 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-67b513e1-ed2c-4c13-bc34-941a3f72868e tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:30.141330 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-67b513e1-ed2c-4c13-bc34-941a3f72868e tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:30.141765 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-67b513e1-ed2c-4c13-bc34-941a3f72868e tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:30.142480 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-67b513e1-ed2c-4c13-bc34-941a3f72868e tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:30.143023 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-67b513e1-ed2c-4c13-bc34-941a3f72868e tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:30.143796 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-67b513e1-ed2c-4c13-bc34-941a3f72868e tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:30.279370 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-67b513e1-ed2c-4c13-bc34-941a3f72868e tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:30.282464 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-67b513e1-ed2c-4c13-bc34-941a3f72868e tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 200 len: 1497 microversion: 2.1 time: 0.152983 Sep 08 14:40:30.283034 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1604/2924] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:30 2022] GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:40:31.169064 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-39db473c-fc46-41bb-98dd-19ac6064afe9 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:31.169586 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-39db473c-fc46-41bb-98dd-19ac6064afe9 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:31.176394 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39db473c-fc46-41bb-98dd-19ac6064afe9 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:31.176997 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39db473c-fc46-41bb-98dd-19ac6064afe9 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:31.177440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39db473c-fc46-41bb-98dd-19ac6064afe9 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:31.178130 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39db473c-fc46-41bb-98dd-19ac6064afe9 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:31.178611 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39db473c-fc46-41bb-98dd-19ac6064afe9 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:31.179549 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-39db473c-fc46-41bb-98dd-19ac6064afe9 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:31.204712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-39db473c-fc46-41bb-98dd-19ac6064afe9 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Going to try to terminate instance {{(pid=127602) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 08 14:40:31.469418 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-39db473c-fc46-41bb-98dd-19ac6064afe9 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "DELETE /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 204 len: 0 microversion: 2.1 time: 0.304394 Sep 08 14:40:31.470156 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1321/2925] 10.222.0.25 () {58 vars in 1239 bytes} [Thu Sep 8 14:40:31 2022] DELETE /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 08 14:40:31.478562 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-13179536-f236-4fa3-8db9-664023638fa3 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:31.479166 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-13179536-f236-4fa3-8db9-664023638fa3 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:31.485529 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-13179536-f236-4fa3-8db9-664023638fa3 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:31.486000 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-13179536-f236-4fa3-8db9-664023638fa3 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:31.486474 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-13179536-f236-4fa3-8db9-664023638fa3 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:31.487149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-13179536-f236-4fa3-8db9-664023638fa3 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:31.487626 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-13179536-f236-4fa3-8db9-664023638fa3 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:31.488046 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-13179536-f236-4fa3-8db9-664023638fa3 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:31.622516 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.policy [None req-13179536-f236-4fa3-8db9-664023638fa3 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127601) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:31.630376 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-13179536-f236-4fa3-8db9-664023638fa3 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 200 len: 1503 microversion: 2.1 time: 0.155770 Sep 08 14:40:31.630971 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1605/2926] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:31 2022] GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 1503 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:32.654440 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-5e9a381c-6e2a-4852-b57b-b712828d963b tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:32.655418 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-5e9a381c-6e2a-4852-b57b-b712828d963b tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:32.662259 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e9a381c-6e2a-4852-b57b-b712828d963b tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:32.662756 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e9a381c-6e2a-4852-b57b-b712828d963b tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:32.663786 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e9a381c-6e2a-4852-b57b-b712828d963b tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:32.664511 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e9a381c-6e2a-4852-b57b-b712828d963b tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:32.664959 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e9a381c-6e2a-4852-b57b-b712828d963b tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:32.665384 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-5e9a381c-6e2a-4852-b57b-b712828d963b tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:32.798661 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-5e9a381c-6e2a-4852-b57b-b712828d963b tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:32.806269 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-5e9a381c-6e2a-4852-b57b-b712828d963b tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 200 len: 1503 microversion: 2.1 time: 0.156181 Sep 08 14:40:32.807052 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1322/2927] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:32 2022] GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 1503 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:33.829032 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-221a9e18-c1ea-4339-8867-6f852895c891 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:33.829876 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-221a9e18-c1ea-4339-8867-6f852895c891 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:33.836219 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-221a9e18-c1ea-4339-8867-6f852895c891 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:33.836705 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-221a9e18-c1ea-4339-8867-6f852895c891 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:33.837146 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-221a9e18-c1ea-4339-8867-6f852895c891 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:33.837842 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-221a9e18-c1ea-4339-8867-6f852895c891 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:33.838507 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-221a9e18-c1ea-4339-8867-6f852895c891 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:33.838936 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-221a9e18-c1ea-4339-8867-6f852895c891 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:33.902289 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-221a9e18-c1ea-4339-8867-6f852895c891 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:33.912349 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [req-346fa369-280e-4f95-ace7-6528f084cc92 req-0bb7424b-da56-407b-83d4-4e37a440447f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "29c8d7e2-687b-4169-95da-4446901ffd51", "name": "network-vif-deleted", "tag": "85bc17ba-5320-42c0-a869-d4017b754772"}]} {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 08 14:40:33.917908 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-221a9e18-c1ea-4339-8867-6f852895c891 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 200 len: 1461 microversion: 2.1 time: 0.092639 Sep 08 14:40:33.918521 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1323/2928] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:33 2022] GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 1461 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 08 14:40:33.922523 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-346fa369-280e-4f95-ace7-6528f084cc92 req-0bb7424b-da56-407b-83d4-4e37a440447f service nova] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:33.922998 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-346fa369-280e-4f95-ace7-6528f084cc92 req-0bb7424b-da56-407b-83d4-4e37a440447f service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:33.923430 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [req-346fa369-280e-4f95-ace7-6528f084cc92 req-0bb7424b-da56-407b-83d4-4e37a440447f service nova] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:33.935136 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.compute.server_external_events [req-346fa369-280e-4f95-ace7-6528f084cc92 req-0bb7424b-da56-407b-83d4-4e37a440447f service nova] Creating event network-vif-deleted:85bc17ba-5320-42c0-a869-d4017b754772 for instance 29c8d7e2-687b-4169-95da-4446901ffd51 on hv-cinder-85646 Sep 08 14:40:33.940134 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [req-346fa369-280e-4f95-ace7-6528f084cc92 req-0bb7424b-da56-407b-83d4-4e37a440447f service nova] 10.222.0.25 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.153936 Sep 08 14:40:33.940716 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1606/2929] 10.222.0.25 () {64 vars in 1317 bytes} [Thu Sep 8 14:40:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 08 14:40:34.940994 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.api.openstack.wsgi [None req-1599ea96-af12-45e2-af71-fa3f1ef1efa6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127602) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:34.941503 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.compute.api [None req-1599ea96-af12-45e2-af71-fa3f1ef1efa6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127602) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:34.948362 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1599ea96-af12-45e2-af71-fa3f1ef1efa6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:34.948843 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1599ea96-af12-45e2-af71-fa3f1ef1efa6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:34.949277 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1599ea96-af12-45e2-af71-fa3f1ef1efa6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:34.949962 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1599ea96-af12-45e2-af71-fa3f1ef1efa6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:34.950443 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1599ea96-af12-45e2-af71-fa3f1ef1efa6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:34.950865 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1599ea96-af12-45e2-af71-fa3f1ef1efa6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:34.981236 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1599ea96-af12-45e2-af71-fa3f1ef1efa6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:34.981712 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1599ea96-af12-45e2-af71-fa3f1ef1efa6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:34.982175 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG oslo_concurrency.lockutils [None req-1599ea96-af12-45e2-af71-fa3f1ef1efa6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127602) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:34.982608 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.objects.instance [None req-1599ea96-af12-45e2-af71-fa3f1ef1efa6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lazy-loading 'fault' on Instance uuid 29c8d7e2-687b-4169-95da-4446901ffd51 {{(pid=127602) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 08 14:40:35.027238 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: DEBUG nova.policy [None req-1599ea96-af12-45e2-af71-fa3f1ef1efa6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '08c895693aa54683951e91d4271c6191', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '307bc84fa6ea41cda154754458eff813', 'project_domain_id': 'default', 'roles': ['reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=127602) authorize /opt/stack/nova/nova/policy.py:199}} Sep 08 14:40:35.035264 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: INFO nova.api.openstack.requestlog [None req-1599ea96-af12-45e2-af71-fa3f1ef1efa6 tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 200 len: 1303 microversion: 2.1 time: 0.098248 Sep 08 14:40:35.035819 dv-cinder-856460-1-iscsi devstack@n-api.service[127602]: [pid: 127602|app: 0|req: 1324/2930] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:34 2022] GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 1303 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 08 14:40:36.066282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f9af8768-edf0-490d-858d-e64a843229ca tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Calling method '>' {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 08 14:40:36.066282 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.compute.api [None req-f9af8768-edf0-490d-858d-e64a843229ca tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] [instance: 29c8d7e2-687b-4169-95da-4446901ffd51] Fetching instance by UUID {{(pid=127601) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 08 14:40:36.066904 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9af8768-edf0-490d-858d-e64a843229ca tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:36.067305 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9af8768-edf0-490d-858d-e64a843229ca tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:36.067769 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9af8768-edf0-490d-858d-e64a843229ca tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:36.068464 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9af8768-edf0-490d-858d-e64a843229ca tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Acquiring lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 08 14:40:36.068921 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9af8768-edf0-490d-858d-e64a843229ca tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 08 14:40:36.069912 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG oslo_concurrency.lockutils [None req-f9af8768-edf0-490d-858d-e64a843229ca tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Lock "9ad88c02-0a33-4de9-8963-1a2c7b0cd0cf" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=127601) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 08 14:40:36.083708 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.wsgi [None req-f9af8768-edf0-490d-858d-e64a843229ca tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] HTTP exception thrown: Instance 29c8d7e2-687b-4169-95da-4446901ffd51 could not be found. Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f9af8768-edf0-490d-858d-e64a843229ca tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Request method failure captured: Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: request: GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 HTTP/1.1 Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept: application/json Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Accept-Encoding: identity Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Connection: close Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Length: 0 Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Content-Type: application/json Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Host: 10.222.0.25 Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: Openstack-System-Scope: None Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: User-Agent: python-urllib3/1.26.12 Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Auth-Token: gAAAAABjGf63izntBoSNvhJPI_fmsgIWS0vzcbmBFO5a1EorMoSh53FWRxht9zuF6-AtdBe-r8DZh1-lfLeu8LN7PbpNf3abTp7sIzfM_BQJeIFS2NQH6wyHcQCuAQoIo4bXYyCv6wHU62sQp4bF6c27CXZbLk0X0dlM3CcZq-C78eNqBkqeHdM Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Id: None Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Domain-Name: None Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Identity-Status: Confirmed Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Is-Admin-Project: True Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Id: default Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Domain-Name: Default Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Id: 307bc84fa6ea41cda154754458eff813 Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Project-Name: tempest-VolumesActionsTest-369921482 Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Role: reader,member Sep 08 14:40:36.086149 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Roles: reader,member Sep 08 14:40:36.087504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Service-Catalog: [{"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2.1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.25:8080", "publicURL": "http://10.222.0.25:8080/v1/AUTH_307bc84fa6ea41cda154754458eff813"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/307bc84fa6ea41cda154754458eff813"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/volume/v3/307bc84fa6ea41cda154754458eff813"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/identity"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25:9696/networking"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/compute/v2/307bc84fa6ea41cda154754458eff813"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.25/placement"}]}] Sep 08 14:40:36.087504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant: tempest-VolumesActionsTest-369921482 Sep 08 14:40:36.087504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Id: 307bc84fa6ea41cda154754458eff813 Sep 08 14:40:36.087504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-Tenant-Name: tempest-VolumesActionsTest-369921482 Sep 08 14:40:36.087504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User: tempest-VolumesActionsTest-369921482-project Sep 08 14:40:36.087504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Id: default Sep 08 14:40:36.087504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Domain-Name: Default Sep 08 14:40:36.087504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Id: 08c895693aa54683951e91d4271c6191 Sep 08 14:40:36.087504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: X-User-Name: tempest-VolumesActionsTest-369921482-project Sep 08 14:40:36.087504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: method: > Sep 08 14:40:36.087504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: exception: Instance 29c8d7e2-687b-4169-95da-4446901ffd51 could not be found. Sep 08 14:40:36.087504 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: action_args: {'id': '29c8d7e2-687b-4169-95da-4446901ffd51'} Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: {{(pid=127601) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 29c8d7e2-687b-4169-95da-4446901ffd51 could not be found. Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 08 14:40:36.088344 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 08 14:40:36.089783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 08 14:40:36.089783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 08 14:40:36.089783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 08 14:40:36.089783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 08 14:40:36.089783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 08 14:40:36.089783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 08 14:40:36.089783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 08 14:40:36.089783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 29c8d7e2-687b-4169-95da-4446901ffd51 could not be found. Sep 08 14:40:36.089783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:40:36.089783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 08 14:40:36.089783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:40:36.089783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 08 14:40:36.089783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 08 14:40:36.089783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 08 14:40:36.089783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 08 14:40:36.089783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 08 14:40:36.089783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 29c8d7e2-687b-4169-95da-4446901ffd51 could not be found. Sep 08 14:40:36.089783 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: ERROR nova.api.openstack.wsgi Sep 08 14:40:36.091013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: DEBUG nova.api.openstack.wsgi [None req-f9af8768-edf0-490d-858d-e64a843229ca tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] Returning 404 to user: Instance 29c8d7e2-687b-4169-95da-4446901ffd51 could not be found. {{(pid=127601) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 08 14:40:36.091013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: INFO nova.api.openstack.requestlog [None req-f9af8768-edf0-490d-858d-e64a843229ca tempest-VolumesActionsTest-369921482 tempest-VolumesActionsTest-369921482-project] 10.222.0.25 "GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51" status: 404 len: 111 microversion: 2.1 time: 0.032845 Sep 08 14:40:36.091013 dv-cinder-856460-1-iscsi devstack@n-api.service[127601]: [pid: 127601|app: 0|req: 1607/2931] 10.222.0.25 () {58 vars in 1236 bytes} [Thu Sep 8 14:40:36 2022] GET /compute/v2.1/servers/29c8d7e2-687b-4169-95da-4446901ffd51 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0)